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ABSTRACT 

The  Operation  with  Economic  Efficiency  (OEE)  Program  in  the  Republic  of 
Korea  Air  Force  (ROKAF)  was  designed  to  reduce  the  cost  and  improve  the  efficiency 
of  all  ROKAF  operations.  The  ROKAF  is  now  faced  with  the  problem  of  developing 
a  more  efficient  pilot  candidate  selection  and  training  program  which  will  improve  pilot 
quality  and  reduce  attrition  rates  and  the  number  of  aircraft  accidents.  The  present 
research  provides  a  starting  point  for  the  development  of  a  more  efficient  pilot 
selection  and  training  program.  If  successful  this  program  should  result  in  the  selection 
of  pilot  candidates  exhibiting  a  significantly  higher  probability  of  completing  the  pilot 
training  program.  This  thesis  also  provides  a  summary  of  aviation  selection  test 
batteries  presently  in  use  by  the  U.S.  Air  Force  and  U.S.  Navy  and  applies  the 
principles  of  decision  theory  in  determining  whether  a  change  in  the  ROKAF's  present 
pilot  selection  system  is  feasible. 
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I.  INTRODUCTION 

A.       BACKGROUND 

Throughout   the   history   of  aviation   there   has   been   constant   concern   with 

selecting   those    individuals   with   both   mental    and   physical   qualities    desirable   for 

optimum  performance  in  flight  environments.    The  intensity  and  quantity  of  research 

performed  over  the  years  in  the  particular  area  of  pilot  selection  highlights  the  fact  that 

the  key  to  combat  air  superiority  is  the  selection  of  high  quality  pilot  candidates. 

During  World  War  II  and  the  Korean  conflict  it  became  further  apparent  that  the 

ability  to  gain  total  tactical  and  strategic  advantage  over  the  enemy  depended  on 

whether  or  not  air  superiority  was  achieved.    Fortunately  the  U.S.  was  able  to  achieve 

this  superiority  in  the  air  during  the  last  three  major  conflicts.   Unfortunately  there  is  a 

growing  consensus  among  military  analysts  that  the  U.S.  may  be  lulling  itself  into  a 

sense  of  complacency.    The  following  quotation  from  a  publication  by  B.  K.  HoUoway 

underscores  the  basis  for  this  growing  complacency:   [Ref  1] 

•  "Twenty-six  million  living  Americans  are  veterans  of  military  service,  and  most 
of  them  have  served  in  wartime.  How  many  of  these  26  million  ever  had  to  face 
an  enemy  who  held  air  superiority  ?  ...  In  all,  probably  no  more  than  one  out 
of  150,  for  after  February  1943  the  U.S.  and  allies  had  undisputed  air 
superiority  in  the  Mediterranean  and  Western  Europe;  after  August  of  1943  we 
had  it  in  the  Pacific...  In  Korea  we  won  air  superiority  twice  -  from  the  North 
Korean  Air  Force  in  the  first  two  months  of  that  war,  and  again  from  the 
Chinese  Air  Force  after  November  1950.  In  South  Vietnam,  our  air  superiority 
has  come  by  default.  In  North  Vietnam  it  has  yet  to  be  seriously  challenged. " 

Today  we  have  reached  an  age  of  strategic  parity  where  Eastern  block 
technological  developments  in  air  combat  systems  are  advancing  at  a  rate  comparable 
with  those  as  the  West.  The  key  in  future  conflicts  thus  becomes  improved  combat 
effectiveness  and  thus,  ultimately,  the  development  of  improved  aircrew  selection  and 
training  programs. 

The  essence  of  these  programs  might  be  summarized  simply  as: 

1)  Using  an  effective  selection  method,  find  the  best  candidate  pilot, 

2)  Provide  hinx'her  with  the  best  and  most  cost  effective  training, 

3)  Significantly  improve  retention  rates  to  maximize  the  training  investment. 


The  first  step,  selecting  the  best  pilot  candidate,  is  not  an  easy  one.  The  key 
questions  become,  "What  mental  and  physical  characteristics  do  we  look  for  ?"  and 
"What  are  our  selection  criteria  ?"  The  answers  to  these  two  basic  questions  have  been 
the  object  of  much  of  the  pilot  candidate  selection  research,  which  will  be  addressed  in 
the  following  section. 

The  ROKAF  is  currently  experiencing  an  excessive  and  unacceptable  attrition 
rate  in  its  pilot  training  programs.  This  problem  is  further  compounded  by  the  fact 
that  the  total  number  of  candidates  admitted  to  the  flight  training  program  has  steadily 
increased  since  the  early  I980's,  while  training  resources  remained  limited.  As  a  result 
the  candidates  who  remain  in  the  program  are  afforded  fewer  hours  of  instruction  in 
the  air  and  thus  do  not  receive  the  desired  quality  or  quantity  of  training. 

In  the  early  1960's  both  the  U.S.  Navy  and  Air  Force  flight  training  programs 
suffered  similar  attrition  problems.  Figure  l.l  provides  a  summary  of  the  attrition 
categories  for  the  respective  services  at  that  time  and  Appendix  A  provides  a  detailed 
description  of  attrition  categories.  The  comparison  of  attrition  rates  between  USN  and 
USAF  presented  in  Figure  1.1  as  a  result  of  various  factors.  USN  had  60.5%  attrition 
rate  and  21.5%  of  that  was  due  to  flight  failure.  Similarly  USAF  had  49.0%  due  to 
flight  failure  out  of  33.8%  attrition.  However,  with  the  significant  improvement  in 
existing  selection  methods  and  the  incorporation  of  new  methods  of  psychological 
screening  both  services  were  able  to  decrease  their  respective  attrition  rates  by  almost 
30%  by  1977  [Ref  2]. 

The  present  research  attempts  to  summarize  some  of  these  improved  selection 
methods  utilized  by  both  the  U.S.  Navy  and  Air  Force  and  presents  recommendations 
based  on  this  summary  for  improving  the  pilot  selection  program  of  the  ROKAF. 

B.       OBJECTIVE 

The  Korean  Air  Force  (ROKAF)  needs  a  more  effective  way  to  select  and  predict 
performance  of  pilot  candidates  for  success  in  primary  flight  training  and  operational 
flight  training  to  maintain  its  air  superiority.  The  purpose  of  this  thesis,  therefore,  is  to 
suggest  a  pilot  selection  system  which  selects  potential  successes  in  flight  training  and 
improves  pilot  quality.  This  research  also  provides  a)  an  overview  of  pilot  selection 
research,  b)  testing  methods,  and  c)  a  related  decision  theory  and  cost  effectiveness 
model. 
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USN 

60.5  % 


USAF 

33.8  % 


Motivational  Attrition 


NOM  18.4% 

(Not  OfTicer 
Material) 

DOR^  42.1% 

(Drop  on  Request 
or  Voluntary 
Withdrawal) 

SIE  22.4% 

(Self 
Initiated 
Elimination) 

MOA  11.4% 

(Manifestation 
of  Apprehension) 

Flight 


FF  21.5% 
(Flight  Failure) 


FD  49.0% 
(Flying  Deficiency) 


Medical 


NPQ  14.9% 

(Not  Physically 
Qualified) 


M  12.3% 
(Medical) 


Academic 


2.0% 

2.9% 

Other 


1.1% 

2.0% 

includes  3.6%  Air  Sickness. 


Figure  1.1     Representative  Navy  and  Air  Force  Pilot  Attrition  Categories  (From  Ref2). 
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II.  U.S.  PILOT  SELECTION  RESEARCH 

A.       WORLD  WARS 

It  was  first  noted  during  WW  I  that  a  significant  number  of  aircraft  accidents 
were  due  not  primarily  to  aircraft  failures  but  to  human  error.  Thus,  based  on  this 
data  France,  Italy  and  England  all  initiated  programs  to  aid  in  the  prediction  of  pilot 
success  during  primary  flight  training.  U.S.  efforts  began  as  early  as  1919.  [Ref  3] 
Early  selection  tests  in  U.S.  were  based  primarily  on  Paper-and-Pencil 
Perceptual/Cognitive  tests  with  psychomotor  devices  and  were  able  to  predict  flight 
performance  of  pilot  candidates  with  resonable  validity. 

The  development  of  selection  tests  continued  between  the  First  and  Second 
World  Wars.  Much  of  this  effort  was  aimed  at  the  development  of  psychomotor  test 
devices  which  simulated  stick  and  rudder  movements  of  the  aircraft. 
Perceptual/ Cognitive  Paper-and-Pencil  tests  likewise  continued  to  be  developed  but 
both  these  and  the  psychomotor  tests  lacked  significant  predictive  validity.  A 
contributor  to  this  fact  was  the  deficiency  in  external  criteria  of  in-flight  performance. 
During  WW  II  the  demand  to  train  large  numbers  of  pilots  in  a  short  amount  of  time 
along  with  increased  training  cost  created  the  necessity  for  minimizing  attrition  due  to 
poor  proficiency.  The  Navy  thus  incorporated  in  its  selection  test  battery  numerous 
items  previously  evaluated  in  the  civilian  pilot  training  program.  In  the  "Pensacola 
1000  aviator  study",  evaluation  was  made  of  the  validity  of  more  than  60  psychological, 
physiological  and  psychomotor  test  items  from  the  Navy  selection  test  battery  utilizing 
over  900  Navy  flight  candidates.  Results  indicated  that  psychological  and  psychomotor 
measures  exhibited  greater  validity  for  the  prediction  of  success  in  flight  training  than 
did  physiological  measures  and  verified  the  vaUdity  and  effectiveness  of  these 
psychological  instruments  and  psychomotor  tests  in  the  prediction  of  flight  success. 
[Ref  3] 

The  US  Army  and  Air  Force  also  developed  their  pilot  selection  testing  battery 
during  WW  II  and  several  useful  methods  such  as  the  Complex  Coordination  test  and 
Divided  Attention  test  are  still  used  in  the  present  selection  test  battery.  The  Complex 
Coordination  test,  similar  to  USN  one,  used  by  the  requires  the  candidate  to  make 
simple  controlled  movements  of  a  stick  and  rudder  in  response  to  patterns  of  visual 
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stimuli.  During  WW  II  it  was  the  most  useful  and. most  highly  weighted  test.  Other 
testing  methods  developed  included  the  Finger  Dexterity,  Rudder  Control  Test  and 
Two  Hand  Pursuit  Test.  Typical  apparatus  test  validity  coefficients  for  pilots, 
navigators  and  bombardiers  are  presented  in  Table  1  [Ref  4].  Note  that  the  test  most 
highly  related  to  pilots  is  the  Complex  Coordination  Test.  The  Discrimination 
Reaction  Time  is  significantly  related  to  navigators  and  bombardiers. 


TABLE  1 

VALIDITY  OF  APPARATUS  TESTS 

(year,  sample  size) 

Complex  Coordination 

Pilots 

Navigators 

Bombardiers 

(1943 
r=.33 

3151) 
rc=.40 

(1942,  1022) 
r=.l7;r^=.24 

(1943,  1829) 
r=.l0;r^=.i3 

Rotarv  Pursuit  with 
Divided  Attention 

(1943 
r=.14 

3146) 
r,=  .i2 

Finger  Dexterity 

(1943 
r  =  .07 

4779) 

(1943,  1021) 
r=.10,  r^=I3 

(1943,  1828) 
r=.l3,  rc=I5 

Discrimination 
Reaction  Time 

(1943 
r=.25 

4779) 
rc=.28 

(1942,  1022) 

r=.27;r^=.35 

(1942,  1829) 
r=.22,  r^  =  .25 

Rudder  Control  Test 

(1943 

r  =  .22 

3146)^ 
r^  =  .30 

Two  Hand  Pursuit 

(1943 
r  =  .27 

1385)     ^ 
(average) 

Two  Hand 
Coordination 

(1943 
r=.31 

4779) 
r^=.35 

(1942,  1022) 
r=.26,  r^=.29 

(1943,  1828) 
r=.09,  r^,=  .12 

r   =  validity  coefficient 

based  on  dichotomous 

pass/fail  training 

criterion. 

r^  =  validity  coefficient  corrected  for  restriction  in  range. 

(From  Reference  4) 
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B.       POSTWAR   RESEARCH 

Psychomotor  Tests  were  used  in  USAF  selection  battery  until  1951  but  were 
deleted  due  to  administrative,  quality  control  and  reliability  problems.  A  subsequent 
study  of  the  revision  of  the  psychomotor  tests  indicated  that  the  use  of  simple  motor 
skill  tests  offered  little  if  any  predictive  value  for  flight  school  success.  It  was 
concluded  that  this  was  due  to:   [Ref  5] 

1)  The  unreliability  of  both  predictor  and  criterion  scores, 

2)  The    use    of  inappropriate    motor    tests    for    the    particular    criteria    being 
evaluated,  and 

3)  The  task  of  flying  an  airplane  is  too  complex  for  simple  motor  skill  tests  to  be 
of  substantial  benefit. 

This  indicated  that  other  more  complex  abilities  were  necessary  for  successful 

pilot  performance  and  that  the  goal  of  future  research  should  be  to: 

1)  Define  the  abilities, 

2)  Quantify  the  basic  behaviors  making  up  the  abilities, 

3)  Devise  accurate  test  measures. 

These  problems  were  considered  continuously  through  the  1970s.  As  a  result, 
currently  the  USAF  has  been  achieving  great  success  in  predicting  flight  performance 
in  its  UPT  program.   This  success  will  be  addressed  in  more  detail  in  Chapter  III. 

The  other  type  test  which  has  been  the  most  widely  and  easily  used  as  a  selection 
test  has  been  the  Perceptual/Cognitive  Paper-and-Pencil  tests.  The  U.S.  military 
service  paper-and-pencil  test  battery  consisted  of:   [Ref  3] 

1)  A  general  intelligence  component  composed  of  verbal  and  quantitative  items, 

2)  Mechanical  comprehension, 

3)  A  spatial  component,  and 

4)  A  background  or  biographical  inventory  composed  of  miscellaneous  subset  of 
items  usually  of  an  historical  nature  known  to  relate  to  aviator  success. 

The  analysis  of  EEG  (Electroencephalogram)  recordings  has  also  received  much 

attention   as   a   possible   predictor   of  aircraft   accidents.     It  has   been  found   that 

individuals  with  abnormal  EEG's  are  involved  in  accidents  at  three  times  the  rate  of 

those  individuals  with  normal  EEG's^  [Ref  3].  As  a  result  of  these  and  similar  findings 

the    Nav7   now    screens    all    new    student    aviators    for   abnormal    EEG's    prior    to 

commencement  of  flight  training. 


These  fmdings  have  resulted  in  the  Danish  Air  Force's  use  of  EEG  recordings  in 
the  selection  of  pilot  candidates. 
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C.       CURRENT  RESEARCH 

The  establishment  of  reliable  criterion  measures  is  a  prime  factor  in  any  selection 
process.  This  continues  to  be  an  extremely  difficult  and  dynamic  task.  For  example, 
the  Nav\'  has  recently  found  that  selection  variables  or  criterion  used  to  predict 
undergraduate  training  success  and  early  aviation  training  performance  measures  may 
not  be  related  or  may  even  be  inversely  related  to  performance  in  the  advanced 
operational  environment  [Ref  3]. 

One  facility  which  has  been  invaluable  in  providing  insight  into  the  establishment 
of  viable  performance  criterion  for  possible  use  in  selection  tests  is  the  Navy  Air 
Combat  Maneuvering  Range  (ACMR).  This  facility  allows  multiple  fighter  aircraft  to 
engage  in  a  tactical  environment,  allowing  the  simulated  employment  of  air  to  air 
weapons  as  a  means  of  providing  training  in  tactical  skills,  weapon  utilization  and 
weapon  envelope  recognition.  Several  problems  arise  from  utilizing  ACMR  generated 
criterion,  however.  First,  the  high  psychological  stress  levels  associated  with  actual 
combat  may  be  absent,  secondly,  individual  performance  is  dependent  on  adversary 
performance.  Thus  an  independent  performance  assessment  is  difficult  if  not 
impossible.  Plans  are  ongoing  to  better  utilize  this  facility  in  understanding  aviator 
skills  which  appear  to  enhance  ACMR  performance. 

Finally,  Perceptual  Psychomotor  Testing  and  Division  of  Attention  Tests  are 
being  improved  by  both  the  U.S.  and  Israeli  Air  Force  respectively.  Psychomotor 
testing  has  already  made  a  unique  contribution  to  the  USAF  UPT  selection  program. 
Israeli  Air  Force  has  likewise  experienced  success  in  predicting  training  success  in  high 
performance  jet  aircraft  using  the  Division  of  Attention  Test    [Ref.  6]. 

We  have  now  discussed  what  testing  methods  have  been  developed  and  exist 
today,  and  have  established  how  important  they  are  in  developing  the  pilot  selection 
program.  We  now  suggest  useful  pilot  selection  methods  for  the  ROKAF  which 
hopefully  will  result  in  considerable  resource  savings  and  significant  improvements  in 
pilot  quality. 


^Approximately  for  2,000  subjects,  the  data  was  significantly  related  to  success  in 
jet  training. 
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III.  AVIATOR  SELECTION  METHODS  FOR  ROKAF 

Although  the  human  resources  and  materials  for  the  flight  training  program  in 
the  ROKAF  have  increased  only  slightly,  the  number  of  pilot  candidates  is  increasing 
every  year  as  the  number  of  Korean  Air  Force  Academy  (KAFA)  cadets  increases. 
Since  the  only  admission  requirement  to  the  KAFA  for  flight  training  is  a  medical 
clearance,  and  most  pass  the  medical  examination,  the  ROKAF  pilot  training 
command  cannot  handle  the  number  of  cadets  and  candidates  from  other  sources 
(AFROTC  and  Navy).  This  increased  number  of  pilot  candidates  has  created  the 
necessity  for  minimizing  the  number  of  candidates  lost  due  to  poor  or  unsatisfactory 
proficiency  in  the  UPT  program;  maximizing  the  probability  that  potential  failures  will 
be  screened  out  prior  to  flight  training. 

The  present  pilot  selection  procedures  of  the  ROKAF  do  not  adequately  take 
into  account  candidates'  mental  abilities.  The  electrocadiogram,  however,  is  being  used 
for  fighter  pilots,  and  an  abnormal  electrocadiogram  will  prohibit  continuation  of  flight 
training.  All  of  the  KAFA  cadets  who  pass  the  medical  examination  enter  the  flight 
training  program  and  have  a  chance  to  fly.  Flight  training  starts  early  before 
graduation  from  the  KAFA  and  ends  about  a  year  after  graduation.  This  system,  with 
tlight  training  begining  early  before  graduation  was  adapted  during  the  I980's. 
However  the  current  system  does  not  appear  to  represent  a  significant  improvement  in 
previous  program.  Cadets  who  fail  as  a  cadet  fmd  it  difficult  to  continue  the  program 
and  those  who  fail  as  an  officer  after  graduation  and  commissioning  fmd  it  difficult  to 
adapt  to  different  specialties.  Candidates  are  divided  into  several  subgroups  in  order  to 
meet  the  limited  training  squadron  resources.  Graduates  of  the  subgroups  enter  the 
next  phase  of  training  with  attrites  returning  to  the  KAFA.  Attritions  continue 
through  the  flight  training  -  from  the  primary  training  program  up  to  the  advanced 
program  over  several  years.  The  only  way  that  candidates  could  be  differentiated  is  on 
some  amount  of  actual  flying  at  the  training  squadron  and  the  failures  would  be  totally 
subjective  based  on  instructors'  feeling,  knowledge  and  their  experiences,  except  for 
voluntary  withdrawal  or  drop  on  request  (DOR). 

The  following  demonstrates  a  useful  way  to  help  in  solving  the  existing  short 
comings  of  the  current  ROKAF  pilot  training  program  and  suggests  pilot  candidate 
selection  methods  which  might  be  used  as  a  starting  point  in  alleviating  these  problems. 
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A.       METHODS 

This  section  suggests  selection  methods  for  the  ROKAF  which  are  currently 
being  used  in  the  U.S.  The  purpose  o[  the  tests  are  to  select  candidates  who  have  a 
high  probability  of  completing  the  flight  training  program.  Multitask  performance  is 
being  used  in  USN  and  it  has  enjoyed  considerable  success  in  predicting  flight  training 
[Ref  7].  The  USAF  uses  a  psychomotor  screening  system  for  selecting  candidates  as 
well  as  identifying  whether  the  candidate  is  suited  for  fighter  pilot  training  [Ref  8]. 
The  USAF  also  suggests  that  an  integrated  model  including  psychomotor  testing  and 
other  available  information  -  AFOQT  scores,  technical  degree,  prior  flying  experience, 
age  of  subjects,  etc.  -  is  better  than  using  only  the  psychomotor  test.  All  three 
methods  use  a  psychomotor  test  since  the  psychomotor  scores  have  been  found  to 
relate  significantly  to  flight  training  pass/ fail  criteria.  As  the  USN  adapted  the  EEG 
recordings  as  a  direct  result  of  the  Danish  Air  Force  as  we  discussed  in  Chapter  II,  the 
ROKAF  may  choose  one  of  these  three  methods  or  a  combination  and  apply  it  to 
ROKAF  cadet  selection  with  some  possible  modifications.  The  following  text  describes 
the  three  methods  which  could  be  used  and  modified  for  the  ROKAF  and  shows  how 
the  predicted  resuhs  may  be  related  to  the  actual  outcomes. 

1.  Multitask  Performance 

The  USN  had  great  success  in  predicting  student  success  in  primary  flight 
training  utilizing  multitask  performance  tests  in  1986  [Ref  7].  A  Psychomotor  Test 
(PMT)  and  a  Dichotic  Listening  Task  (DLT)  were  administered  under  single-task  and 
multitask  conditions  separately  and  in  combination.  Testing  procedures  were  as 
follows:   [Ref  7] 

1)  Psychomotor  Test  (PMT) 

•  The  PMT  requires  subjects  to  simultaneously  center  two  visually 
presented  cursors  on  fixed  targets  displayed  on  a  CRT  screen.  The  test 
performance  measurements  were  the  accumulated  X,  Y.  and  Z  axis  error 
distances  between  the  target  and  the  cursor  point.  A  diagram  of  the 
experimental  apparatus  of  Psychomotor  Task  (PMT)  is  depicted  in  Figure 
3.1  . 

2)  Dichotic  Listening  Task  (DLT) 

•  The  DLT  first  requires  subjects  to  attend  to  auditory  information 
presented  to  one  ear,  while  ignoring  similar  information  presented  to  the 
opposite  ear,  and  then  after  considering  an  auditory  cue,  to  switch 
attention  rapidly  to  the  previously  attended  ear,  or  maintain  attention  to 
the  previously  attended  channel.  The  modified  Dichotic  Listening  task 
(DLT)  [Ref  9]  consisted  of  letter-digit  string  sets  of  24  listening  trials  for 
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Figure  3.1     Experimental  Apparatus  (adapted  from  Ref.7). 

each  of  3  counter-balanced  response  modalities.  Written,  keypad  and 
verbal  response  conditions  were  employed.  The  subjects  were  instructed 
to  direct  attention  to  one  ear  while  ignoring  the  other,  and  correctly 
report  the  nine  digits  per  trial.  An  example  DLT  trial  is  depicted  in 
Figure  3.2  .  The  single-task  performance  measure  for  tiie  DLT  was 
simply  the  number  of  correct  responses  per  24  trials. 

3)       Multitask  Performance 

•  For  the  multitask  condition,  subjects  performed  two  sessions  of  the  DLT 
and  PMT  simultaneously.  Performance  measures  for  the  PMT  and  DLT 
in  the  multitask  conditions  were  identical  to  those  of  the  single-task 
conditions  except  that  multitask  DLT  performance  was  based  on  12 
rather  than  24  trials. 

The  tests  were  performed  in  order  such  as: 

1)  Psychomotor  Test  (PMT), 

2)  Dichotic  Listening  Task  (DLT)  and 
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Figure  3.2    DLT  Trial  Examples  (adapted  from  Ref  7). 

3)       Vlultitask,  that  is,  PMT  and  DLT  simultaneously. 

To  validate  the  test  criteria,  subjects'  flight  performance  were  tracked  until 
completing  pilot  training.  Table  2  shows  the  mean  and  pass/fail  correlation  of  each 
test  measure.  Correlations  between  each  testing  measure  are  presented  in  Table  3  .  As 
you  can  see  in  Table  2  ,  testing  measures  1,  5,  9,  and  11  are  significantly  related  to  the 
flight  training  pass/fail  criteria.  Table  3  presents  a  correlation  matrix  of  test  measures. 
2.  Psychomotor  Test 

The  USAF  has  utilized  psychomotor  testing  in  relation  to  pilot  selection.  The 
USAF  primary  objective  has  been  an  attempt  to  develop  reliable  and  valid  test  devices 
and  validation  of  test  scores.  In  1969,  the  Air  Force  Human  Resource  Laboratory 
(AFHRL)  attempted  to  determine  the  reliability  of  two  computer-based  psychomotor 
tests  (i.e.,  Two-Hand  Coordination  and  Complex  Coordination)  [Ref  8].  Both  tests 
were  employed  to  measure  the  subject's  abihty  to  coordinate  movements.  Subjects 
were  from  Air  Force  Academy  (AFA),  Air  Force  Reserve  Officer  Training  Corps 
(AFROTC),  and  Officer  Training  School  (OTS).  The  largest  group  of  the  sample  was 
AFROTC  and  additional  subjects  from  Williams  AFB  were  used  for  validation  of 
testing  results. 
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TABLE  2 

DESCRIPTIVE  STATISTICS  FOR  SINGLE  AND  MULTITASK 

MEASURES  AND  SELECTION  TESTS 

(percent  correct) 

Test  Measure 

Mean 

Pass;  Fail 

1.  PMT  Session  1,  X,Y,Z  error  combined 

19116.36 

-.283 

2.  PMT  Session  2,  X,Y,Z  error  combined 

11541.20 

-.028 

3.  PMT  Session  3,  X,Y,Z  error  combined 

9896.90 

-.241 

4.  PMT  Session  4,  X,Y,Z  error  combined 

9550.82 

-.120 

5.  DLT  written  response,  No.  correct 

211.48(97.9%) 

.360 

6.  DLT  keypad  response,  No.  correct 

210.60  (97.5%) 

.162 

7.  DLT  vocal  response.  No.  correct 

207.52(96.1%) 

.244 

8.  PMT  X,Y,Z  error  (with  keypad  DLT) 

21816.92 

.036 

9.  DLT  keypad  response  (with  PMT) 

102.34  (94.8%) 

.413 

10.  PMT  X,Y,Z  error  (with  vocal  DLT) 

14038.10 

-.002 

11.  DLT  vocal  response  (with  PMT) 

101.44(93.9%) 

.395 

12.  AQT  standard  score 

5.66 

.172 

13.  FAR  standard  score 

6.80 

.361 

**  Measure   1-7     :  single-task 

**  Measure  8-11    :  multitask 

**  Measure  12,13  :  selection  test 

(From  Reference  7) 

Testing  procedures  were  as  follows:  [Ref  8] 

1)       Two-Hand  Coordination 

•  The  first  of  the  tests  presented  a  triangular-shaped  target  and  a  cross- 
shaped  pipper  on  the  CRT  (Figure  3.3).  The  computer  moved  the  target 
in  an  elliptical  path  and  with  varying  speeds  -  faster  near  the  4  o'clock 
and  slower  near  the  11  o'clock  position.  The  subjects  were  instructed  to 
use  both  joysticks  simultaneously  (one  is  for  up-down  and  the  other  is  for 
left-right)  in  a  coordinated  manner  to  move  the  pipper,  keeping  it  as  close 
as  possible  to  the  target  on  the  CRT.  The  computer  measured  absolute 
distance  from  the  pipper  to  the  target.    The  horizontal  error  (Xj)  and 
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TABLE  3 

CORRELATION  MATRIX  FOR  SINGLE  AND  MULTITASK 
MEASURES  AND  SELECTION  TESTS 

Measure     1 
1 

2 

3 

4         5         6         7         8         9         10       11       12 

I 

2 

.401 

3 

.727 

.544 

4 

.520 

.667 

.838 

5 

-.626 

-.131 

-.694 

-.443 

6 

-.616 

-.490 

-.523 

-.453    .540 

7 

-.494 

-.182 

-.407 

-.325    .708    .544 

8 

.393 

.500 

.501 

.597  -.385  -.503  -.612 

9 

-.708 

-.244 

-.782 

-.508    .869    .588    .563  -.348 

10 

.526 

.298 

.532 

.443  -.531  -.468  -.669    .873  -.445 

11 

-.737 

-.243 

-.732 

-.457    .913    .540    .644  -.363    .887  -.529 

12 

-.157 

-.067 

.060 

-.021    .058    .104    .147    .048    .116    .037    .058 

13 

-.368 

-.239 

-.223 

-.287    .050    .144    .223    .254    .144    .230    .099    .511 

**  See  Table  2  for  description  of  numbered  variables. 

(From  Reference  7) 

vertical  error  (Y^  measurement  were  accumulated  during  5-min  test 
period. 

2)       Complex  Coordination 

•  The  second  test  presented  a  set  of  cross-hairs  centered  on  the  CRT,  a  dot- 
shaped  pipper,  and  a  thin  vertical  bar  at  the  bottom  of  the  CRT  (Figure 
3.4).  The  subject  controlled  the  pipper,  both  horizontally  and  vertically, 
using  the  floor-mounted  joystick.  The  control  responses  were  the  reverse 
of  what  is  traditionally  required  on  aircraft  to  reduce  the  advantage  of  a 
subject  with  prior  flying  experience.  The  subjects  were  instructed  to  use 
the  joystick  to  center  the  pipper  horizontally  and  vertically  on  the 
intersection  of  the  cross-hairs  and  simultaneously  press  the  appropriate 
rudder-style  pedal  to  center  the  "rudder  bar"  over  the  lower  part  of  the 
vertical  cross-hair.  The  computer  measured  in  inches  and  accumulated 
during  5-min  test  period  the  the  absolute  distance  from  the  pipper  to  the 
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Figure  3.3    Two-Hand  Coordination  Test  Depiction  (adapted  from  Ref.7). 

intersection  of  the  cross-hairs  and  from  the  rudder  bar  to  the  vertical 
cross-hair.  The  three  scores  obtained  from  this  test  were  the  cumulative 
horizontal  error  (X2),  vertical  error  (Y2),  and  horizontal  error  for  rudder 
bar  (Z2). 

Table  4  presents  five  error  scores  (Xj,  Yj,  X2,  Y2,  Z2)  for  each  subject. 

These  data  show  that  the  Two-Hand  Coordination  test  was  more  difficult  than  the 

Complex  Coordination.    There  might  also  be  some  common  information  about  the 

subjects   since  all  five  test   scores  measured  psychomotor  ability.     Table  5   shows 

intercorrelations  across  the  tests.    Two  psychomotor  tests  were  measuring  different 

abilities  with  very  little  in  common  between  the  tests  as  indicated  in  Table  5  .    To 

validate  the  psychomotor  testing  scores,  the  mean  scores   of  UPT  graduates  and 

eliminees  were  compared  following  completion  of  flight  training.    Table  6  presents  the 
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Figure  3.4    Complex  Coordination  Test  Depiction  (adapted  from  Ref.7). 

mean  of  the  psychomotor  scores  among  three  categories:  UPT  graduates,  ail  UPT 
eliminees,  and  UPT  flying  deficiency  eliminees.  All  five  scores  had  significant 
differences  between  graduates  and  either  category  of  eliminees.  Note  that  for  all  three 
of  the  Complex  scores  (X2,  Y2,  and  Zj)  were  significantly  worse  than  the  graduates' 
means.  These  results  were  used  to  identify  subjects  more  likely  to  be  fighter  pilots. 
Figure  3.5  presents  this  result  of  all  five  means  for  both  fighter-recommended  (FR) 
and  non-FR  students.  The  FR  students  had  significantly  better  scores  than  the  non- 
FR  students.  The  psychomotor  screening  equation  was  fmaljy  developed  and  the  most 
useful  screening  score  based  on  psychomotor  ability  can  be  computed  from  the 
following  equation  [Ref  8]: 
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TABLE  4 

DESCRIPTIVE  STATISTICS 

(inch) 

Test  Score 

Mean 

SD 

Range 

Two-Hand  Coordination 

XI  (Horizon) 

14,709 

5,400 

428  -  65,268 

Yl  (Vertical) 

16,748 

5,538 

5,687  -  57,292 

Complex  Coordination 

X2  (Horizon) 

4,514 

6,520 

326-  71.040 

Y2  (Vertical) 

3,860 

6,445 

486  -  76,320 

Z2  (Rudder) 

5,580 

6,018 

167-71,040 

(From  Reference  8) 

TABLE  5 

CORRELATION  MATRIX 

Xl 

Yl 

^2 

^2 

XI  (Horizon) 

Yl  (Vertical) 

.87 

X2  (Horizon) 

.20 

.25 

Y2  (Vertical) 

.19 

.23 

.86 

Z2  (Rudder) 

.20 

.24 

.77 

.71 

(From  Reference  8) 

Yi  =  a  +  (b^   X  XI-)  +  (b2  X  Y2i) 

where  Y-     =  psychomotor  screening  score  of  /th  subject 
a      =  constant  (0.95648) 
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TABLE  6 

MEAN  PSYCHOMOTOR  SCORES  BY  LPT  OUTCOME 

^l 

Yl 

^2 

^2 

Z2 

Graduate  Means  (N  =   1,348)   14,315 

16,341 

3,559 

2,058 

4,725 

All  Elim.  Means  (N  =  377)       15,829 

17,621 

4,936 

4,173 

6,678 

Flving  Deficiency  (FD) 

16,302 
Elim.  Means  (N  =  199) 

18,007 

5.593 

4,702 

7,580 

(From  Reference  8) 

bj     =  weight  of  psychomotor  XI  score  (-.0000075748) 

b2     =  weight  of  psychomotor  Y2  score  (-.000020375) 

Xlj  =  psychomotor  Xj  score  of /th  subject 

Y2'  =  psychomotor  Y2  score  of /th  subject 

This  equation  provides  a  screening  score  based  on  psychomotor  ability.    These  scores 
could  be  used  in  a  selection  system  by  rank  ordering;  and  the  best  candidates  could  be 
selected  to  fill  the  required  training  quotas. 
3,  Integrated  Selection  System 

Although  psychomotor  testing  scores  are  significantly  related  to  pass/fail 
criteria  in  UPT,  the  ideal  pilot  selection  method  would  consider  all  available 
informations  on  the  subjects. 

Current  operational  USAF  pilot  selection  criteria  consider  information  from 
the  five  composite  scores  of  the  AFOQT,  age  at  the  start  of  UPT,  possession  of  a 
college  technical  degree,  and  possession  of  a  private  pilot's  license  or  completion  of  a 
USAF  light  aircraft  flying  program  [Ref  8].  Table  7  lists  the  correlations  of  all 
available  measures  with  the  UPT  pass/fail  criterion  for  all  reasons  (FY79  to  FY83). 

The  integrated  model  was  developed  based  on  the  USAF  psychomotor  testing. 
Because  pilot  candidates  are  chosen  differently,  depending  on  their  source  of 
commission,  three  different  Integrated  Pilot  Candidate  Selection  Models  (IPCSMs) 
were  developed. 

The  three  IPCSMs  equations  were  developed  and  will  produce  a  score 
corresponding  to  the  probability  of  UPT  success  using  the  following  equations  : 
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Figure  3.5     Mean  Psychomotor  Scores  by  FR/non-FR. 


26 


TABLE  7 

CORRELATIONS  OF  IPCSM  PREDICTIONS  WITH  UPT  OUTCOME 

Predictors                        Correlation  with  UPT 

Sample  Sizes 

Pass/ Fail 

FSP  Pass;  Fail 

.137 

1,534 

FSP  Final  Grade 

.271 

1,534 

FSP  Sum  of  Grades 

.303 

1,534 

AFOQT  Pilot  Score 

.158 

4,460 

AFOQT  Navigator  Tech  Score 

.148 

4,460 

AFOQT  Academic  Score 

.080 

4,577 

AFOQT  Verbal  Score 

.007 

4,576 

AFOQT  Quantitative  Score 

.138 

4,577 

Psychomotor  XI  Score 

-.118 

1,918 

Psychomotor  Yl  Score 

-.099 

1,919 

Psychomotor  X2  Score 

-.153 

1,906 

Psychomotor  Y2  Score 

-.181 

1,910 

Psychomotor  Z2  Score 

-.146 

1,916 

Age 

-.120 

8,438 

Gender 

.033 

8,438 

Race 

.110 

8,292 

Possession  of  Tech  Degree 

.111 

8,183 

{From  Reference  8) 

1)       IPCSM  I  (OTS  Model) 


Yi  =  a    +  (bj   X  x\-)  +  (b^  X  Y2i)  +  (bj  x  age^) 
+  (b^  X  AFOQTP-)  +  (bj  x  AFOQTN^) 
+  (bg  X  FSP1-)  +  (b^  X  FSP2-) 


where  Yj 
a 
b, 


=  IPCSM  I  score  for  /th  subject 

=  constant  (0.10192) 

=  weight  for  psychomotor  XI  score  (-.000015211) 
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b2  =  weight  for  psychomotor  Y2  score  (-.000010618) 

bj  =  weight  for  age  of  subject  (-.0012) 

b^  =  weight  for  AFOQTP/AFOQT  pilot  score  (0.001 12) 

bj  =  weight  for  AFOQTN/AFOQT  navigator  score  (0.00029067) 

b^  =  weight  for  FSPl/FSP  final  grade  (0.08249) 

b^  =  weight  for  FSP2/FSP  sum  of  grades  (0.01923) 

XI j  =  psychomotor  XI  score  of  zth  subject 

Y2-  =  psychomotor  Y2  score  of  /th  subject 

age-  =  age  at  start  of  UPT  of /th  subject 

AFOQTP-    =  AFOQT  pilot  composite  score  of /th  subject 

AFOQTN'i  =  AFOQT  navigator  composite  score  of /th  subject 

FSP1|  =  FSP  fmal  grade  of /th  subject 

FSP2-  =  FSP  sum  of  grade  of /th  subject 


2)     IPCSM  II  (AFROTC  Model) 

Yi  =  a    +  (b^   X  xi^)  +  (b^  X  Y2i)  +  (b3  x  age^) 
+  (b^  X  AFOQTP-)  +  (bj  x  AFOQTN-) 

where  Y-  =  IPCSM  II  score  for  /th  subject 

a  =  constant  (1.72485) 

bj  =  weight  for  psychomotor  XI  score  (-.0000030409) 

b2  =  weight  for  psychomotor  Y2  score  (-.000022526) 

b^  =  weight  for  age  of  subject  (-.04517) 

b^  =  weight  for  AFOQTP/AFOQT  pilot  score  (0.00124) 

b5  =  weight  for  AFOQTN/AFOQT  navigator  score  (0.00225) 


3)     IPCSM  III  (AFA  Model) 

Y—  a  +  (bj   X  xi^)  +  (b2  X  Y2-)  +  {b^  x  age^)  +  (b^  x  tech^) 

where  Y-  =  IPCSM  III  score  of /th  subject 

a  =  constant  (1.43056) 

bj  =  weight  for  psychomotor  XI  score  (-.0000050818) 

b2  =  weight  for  psychomotor  Y2  score  (-.0000072309) 

bj  =  weight  for  age  of  subject  (-.02111) 

b^  =  weight  for  possession  of  technical  degree  (.08756) 

tech^  =  1  if  /th  subject  has  a  technical  degree,  0  otherwise 
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Figure  3.6    Mean  IPCSM  Scores  by  LPT  outcome. 
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The  mean  of  the  IPCSM  scores  are  graphically  presented  in  Figure  3.6  .  The 
difTerences  between  the  means  were  significant  and  generated  the  better  prediction  than 
using  the  psychomotor  test  only.  One  way  to  select  candidates  is  to  accept  only  those 
candidates  who  obtain  a  screening  score  above  a  predetermined  cutoff.  The  cutoff 
score  would  be  determined  by  the  quota  required  and  attrition  rate.  The  higher  the 
cutoff  score  is  set,  the  greater  the  impact  on  attrition  in  pilot  training  program.  The 
FY83  UPT  sample  was  used  to  cross-validate  the  three  IPCSMs  and  resulted  in  the 
IPCSM  scores  were  significantly  correlated  with  UPT  outcome.  Since  the  "Whole 
Person  Concept"  is  imbedded  in  this  multi-dimensional  IPCSM  approach  to  pilot 
selection,  the  integrated  selection  method  has  better  predictors  than  the  psychomotor 
test  only  does. 

As  we  can  see  through  the  Chapter,  all  three  pilot  selection  methods  include  a 
psychomotor  testing  which  means  psychomotor  testing  scores  are  significantly  different 
between  the  successes  and  the  failures.  It  should  have  the  same  result  if  psychomotor 
tests  were  applied  to  ROKAF.  The  only  thing  different  between  U.S.  and  ROKAF 
would  be  the  cutoff  score  as  a  different  required  quota.  The  ROKAF  decision  maker, 
however,  must  know  the  advantages  of  using  an  experimental  testing  method  in  terms 
of  their  utilities  or  cost  effectiveness.  Chapter  IV  provides  one  way  to  make  a  decision 
and  evaluate  the  outcomes. 
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IV.  EVALUATION  OF  TESTING  iMETHOD 

This  chapter  discusses  how  to  make  appropriate  decisions  which  maximize  an 
overall  Measure  of  Effectiveness  (MOE)  or  Utility  which  might  be  used  in  a  pilot 
selection  program.  The  final  goal  of  personnel  selection  programs  has  traditionally 
been  to  achieve  maximum  accuracy  of  measurement  and  efficiency  of  prediction. 
Decision  theory  recognizes  that  the  outcomes  of  predictions  are  of  primary  importance 
to  individuals  and  organizations.  As  discussed  in  Chapter  III,  for  example,  Korean 
AFA  cadets  are  commissioned  as  officers  after  four  years  of  attendance.  Some  fail 
either  in  undergraduate  or  operational  flight  training.  This  results  in  high  attrition 
cost,  dissatisfaction,  lowered  morale,  and  a  number  of  officers  totally  unneeded  in  the 
service.  Since  the  measurement  and  prediction  are  simply  technical  components  of  a 
system  designed  to  make  decisions,  we  have  to  define  the  iMOE  before  making  any 
decisions.  There  are,  however,  a  number  of  possible  MOEs  in  most  decision  problems. 
For  example,  there  would  be  three  possible  MOEs  in  a  pilot  selection  problem: 

1)  Costs  -  selection,  attrition  and  training, 

2)  Pilot  quality,  and 

3)  Probability  that  the  quota  will  be  satisfied. 

Thus  decision  outcomes  should  be  evaluated  in  terms  of  their  consequences  for 
individuals  and  organizations.  That  is,  in  terms  of  their  overall  MOE  or  Utility.  This 
provides  us  with  a  consistent  way  in  which  to  combine  different  figures  of  merit  or 
subjective  and  judgemental  factors,  and  probabilistic  aspects  of  outcomes  into  an 
overall  MOE. 

To  construct  and  evaluate  a  decision  model,  we  have  to  fmd  parameters  which 
function  in  pilot  selection  outcomes.   The  major  factors  are  [Ref  10]: 

a)  the    base    rate    (the    proportion    of   persons    currently    accepted    who    are 
satisfactory) 

b)  the  selection  ratio  (the  proportion  of  applicants  accepted)  and 

c)  the  validity  coefficient  (the  predictor  criterion  correlation). 

Taylor  and  Russell  [Ref  10]  published  a  series  of  tables  which  illustrated  that  the 
value  of  a  test  was  a  function  of  the  three  considerations  listed  above. 
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Figure  4.1     Four  possible  outcomes  of  personnel  selection  (adapted  from  Ref  11). 

The  possible  outcomes  of  personnel  selection  programs  are  illustrated  in  Figure 
4.1  .  This  figure  depicts  a  bivariaie  plot  for  a  selection  test  with  a  validity  of  .50.  The 
horizontal  axis,  SB',  represents  a  particular  base  rate  of  satisfactory'  criterion 
performance,  in  this  case,  50  %.  The  vertical  axis,  CC,  denotes  the  cutting  score  for  a 
50  %  selection  ratio.  Thus,  Quadrant  A  contains  a  number  of  satisfactory  subjects 
above  the  cutting  score  (valid  positives,  VP),  Quadrant  B  contains  unsatisfactory 
subjects  above  the  cutting  score  (false  positives,  FP),  Quadrant  C  contains 
unsatisfactory  subjects  below  the  cutting  score  (valid  negatives,  VN),  and  Quadrant  D 
contains  satisfactory  subjects  below  the  cutting  score  (false  negatives,  FN).  [Ref  11] 
Once  the  cutoff  score  is  set,  a  definite  number  of  predictions  of  success  and  failure  are 
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made.    As  Figure  4.1  shows,  the  total  number  of. predictive  positives,  N--,  may  be 
calculated  from  : 


Npp  =  VP  +  FP 

Similarly,  the  total  number  of  predictive  negatives,  N  ^,  is  given  by: 

Np„  =  FN  +  VN 

The  total  number  of  actual  positives,  N^  ,  is  given  by: 

N„-  =  FN  +  VP 
ap 

And  the  total  number  of  actual  negatives,  N^^^,  is  calculated  by: 

N^n  =  VN  +  FP 

all 

The  probability  of  a  valid  positives,  P(VP),  is  thus: 

P(VP)  =  VP/(N^p  +  N^^)  =  VP/TN 

where  TN  is  the  total  number  of  candidates.  Figure  4.2  presents  the  probabilities  and 
their  relationships. 

When  outcomes  are  expressed  as  probabilities  they  are  referred  to  as  "hit  rates". 
The  probabilities  or  proportions  of  actual  positives  that  exist  in  the  total  sample  of 
candidates  is  called  the  "Base  Rate"  (BR).-'  The  probability  may  be  calculated  directly 
BR  =  N^  /TN  or  from  the  probabilities  of  two  outcomes  BR  =  P(FN)  +  P(VP)  as  in 
Figure  4.2  .  The  BR  has  reference  to  the  base  rate  of  the  positive  class  and  so  the 
probability  or  base  rate  of  negative  in  the  total  sample  is  given  :  1  —  BR. 

Selection  measures  are  most  useful  when  BRs  are  about  .50  [Ref  10].  As  the 
BR  departs  radically  in  either  direction  from  .50,  the  benefit  of  an  additional  predictor 
becomes  questionable. 


^The  proportion  of  candidates  who  would  be  successful  in  flight  training  without 
use  of  the  test. 
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Actual  Diagnosis 

Test  Diagnosis 

Total 

Positive                        Negative 

Positive 

P(VP)  =  VP/TN           P(FN)=FN/TN 
valid  positive  rate          false  negative  rate 
(Proportion  of               (Proportion  of 
positives  called               positives  called 
positive)                         negative) 

BR  =  N.p/TN 
Base  Rate 
(Probability  of 
actual  positives) 

Negative 

P(FP)  =  FP/TN           P(VN)  =  VN/TN 
false  positive  rate          valid  negative  rate 
(Proportion  of              (Proportion  of 
negatives  called              negatives  called 
positive)                         negative) 

1  -  BR  = 

Nan/TN 
(Probability  of 
actual  negatives) 

Total 

SR=Npp;(Npp  +  Np^)   1-SR  = 

Selection  Ratio                ^pn-'^^pp  +  ^pn) 
(Probability  of              (Probability  of 
predicted  positives)       predicted  negatives) 

• 

Figure  4.2    Outcomes  Expressed  as  Probabilities. 

B.       SELECTION  RATIO  (SR) 

The  probability  or  proportion  of  predicted  positives  among  the  total  sample  of 
candidates  is  called  the  "Selection  Ratio"  (SR)."*  This  probability  may  be  calculated 
directly  as  follows: 


SR  =  Npp,-(Npp  +  Np„) 


The  proportion  of  candidates  selected  with  the  use  of  test. 
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or  from  the  probabilities  of  two  outcomes  such  that  - 

SR  =  P(VP)  +  P(FP) 

Whenever  a  quota  exists  on  the  total  number  of  candidates  that  may  be 
accepted,  the  selection  rate  becomes  more  important.  As  the  SR  approaches  I.O,  for 
example,  which  means  all  candidates  must  be  selected,  it  will  be  dominated  by  other 
alternatives  and  as  the  SR  approaches  zero,  it  dominates  all  other  alternatives  which 
means  a  decision  maker  can  afford  to  be  selective.  Cronbach  and  Glesser  [Ref  12] 
have  shown  that  when  the  frequency  distributions  of  the  two  groups  (accept/reject)  are 
plotted  separately  along  the  same  base  line,  the  optimum  cutting  score  for 
distinguishing  between  the  two  groups  will  occur  at  that  point  where  the  two 
distributions  intersect.  However,  in  most  practical  selection  situations,  the  decision 
maker  is  not  free  to  vary  selection  ratios. 

C.       VALIDITY  COEFFICIENT 

Evaluating  the  benefit  obtained  from  tests  is  of  considerable  practical  interest  and 
has  been  a  problem  in  industrial  psychology.  The  relation  of  benefit  to  validity  has 
long  been  regarded  as  an  important  question.  Most  attempts  to  evaluate  benefits  have 
focused  on  the  validity  coefficient.  As  at  least  five  formulas  for  interpreting  the  validity 
coefficient  have  been  given,  we  face  the  problem  of  choosing  between  them  or  of 
modifying  them.   [Ref,  12] 

The  validity  coefficient  is  usually  defmed  as  the  correlation  of  test  score  with 
outcome  or  criterion  score.  We  shall  use  r^^  in  our  discussion  because  the  decision 
maker  is  interested  in  predicting  the  evaluated  outcome,  that  is,  the  outcome  expressed 
in  utility  units.  The  rule  having  the  longest  history  translates  the  validity  coefficient 
r^^^  into  the  "index  of  forecasting  efficiency"  [Ref.  12], 


E  =  J  \  -  {\  -  r\y)  =  I  r^y  I 


Although  the  validity  coefficient  r^^^  is  a  useful  index,  there  are  many  ways  in 
which  it  falls  short  of  accurately  predicting  success.  From  a  practical  point  of  view,  the 
number  of  correct  decisions  made  by  a  psychological  test  or  assessment  is  more 
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important  than  the  degree  of  association  which  exists  between  predicted  and  obtained 
scores  [Ref.  12].  When  two  variables  are  expressed  in  dichotomous  form  (success- 
failure,  select-reject)  as  in  Figure  4.1  ,  the  correlation  between  them  is  given  by  the 
point-biserial  "r  coefficient".  A  simplified  computational  formula  for  the  r  coefficient 
is:   [Ref  13] 

P(VP)  -  BR  X  SR 
^P  [BR(1  -  BR)SR{1  -  SR)]  ^'^ 


The  value  of  r    is  computed  knowing  only  three  values: 

1)  The  probability  of  a  valid  positive,  P(VP), 

2)  The  base  rate,  BR,  and 

3)  The  selection  ratio,  SR. 

The  computational  formula  for  the  r   coefficient  can  be  solved  for  the  probability 


of  valid  positives: 


P(VP)  =  BR  X  SR  +  t^JbK{\  -  BR)SR(1  -  SR) 


This  expression  is  especially  useful  in  evaluating  the  outcomes  of  test  prediction. 
Given  the  validity  of  a  test  r  ,  the  base  rate,  BR,  and  the  proportion  of  candidates  to 
be  selected,  SR,  it  is  possible  to  completely  specify  the  outcomes  of  predictions. 

Taylor  and  Russell  [Ref.  14)  denounced  the  practice  of  evaluating  the 
effectiveness  of  predictor  variables  solely  on  the  basis  of  their  validity  coefficients. 
They  prepared  tables  for  determining  the  effectiveness  of  a  selection  test  by  estimating 
the  proportion  of  selectees  that  are  considered  satisfactory  when  the  test  validity,  base 
rate,  and  selection  ratio  are  known  or  assumed. 

D.      DECISION  STRATEGIES 

There  are  mainly  two  different  decision  strategies  which  are  single-stage 
(nonsequential)  and  multi-stage  (sequential)  [Ref.  12].  The  major  difference  between 
these  two  strategies  is  when  the  terminal  decision  is  made.  The  general  decision 
process  is  illustrated  schematically  in  Figure  4.3  . 
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Figure  4.3    Schematic  View  of  a  Decision  Process  (adapted  from  Ref  12). 

1.  Single-Stage  Strategy 

A  test  or  test  battery  is  employed  to  all  candidates.  Based  on  a  predetermined 
cutting  score,  the  terminal  decision  is  made.  Individuals  are  either  selected  or  rejected 
and  no  further  decisions  are  made.  This  strategy  corresponds  to  selecting  candidates 
randomly  or  by  a  medical  examination  only  in  pilot  selection. 

2.  Multi-Stage  Strategy 

Terminal  decisions  are  made  only  from  the  view  point  of  a  decision  maker  in 
multi-stage  strategy.  The  cutting  score  on  a  test  battery  may  be  used  to  make 
investigatory  decisions.  This  decision  calls  for  additional  information,  dictating  what 
test  or  procedure  vdU.  be  used  to  gather  that  information.  This  then  leads  to  a  further 
decision.  The  investigatory  decisions  may  continue  through  a  number  of  stages  of 
subsequent  testing  until  terminal  decisions  are  made  regarding  all  applicants.  This  type 
of  decision  process  is  used  to  select  pilot  candidates  using  one  or  more  experimental 
testing  method  until  a  decision  maker  is  satisfying. 
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E.       THE  EXPECTED  UTILITIES  (EU) 

As  discussed  at  the  beginning  of  this  Chapter,  utility  provides  a  consistent  way  to 
compare  each  different  selection  method.  The  comparison  would  be  accomplished  by 
expected  utility,  i.e.,  maximizes  the  expected  utility  across  the  possible  outcomes.  If 
the  four  possible  outcomes  such  that  valid  positive,  false  positive,  valid  negative  and 
false  negative  have  utility  U^  ,  L^  ,  U^.^  and  \1^^  respectively,  then  the  expected 
utility  can  be  described  generally: 

EU    =  U^p  X  P(VP)  +  Uf-p  X  P(FP)  4-  U^n  X  P(VN)  +  U^^  x  P(FN)  -  Uy 

where  Uj  is  the  utility  of  testing  method  employed.  Under  different  decision  strategies, 
the  expected  utility  should  be  considered  separately  so  that  we  can  figure  out  which 
decision  strategy  is  better  with  which  testing  method  such  that  expected  utility  for  all 
possible  outcomes  can  be  maximized. 

A  number  of  different  decision  strategies  can  result  from  combining  different 
testing  procedures.  In  the  ROKAF  selection  problem,  consideration  in  given  to  two 
simple  decision  strategies,  i.e.,  single-stage  decision  with  random  or  medical 
examination,  or  double-stage  decision  with  medical  and  one  experimental  test.  This  is 
based  on  the  assumption  that  ROKAF  has  only  two  alternatives. 

1.  Single-Stage  Decision  with  Random  Selection 

If  the  Korean  Air  Force  uses  a  random  selection  process,  providing  that  the 
SR,  BR,  number  of  applicants  N,  then  the  expected  utility  is  going  to  be: 

EU  =  U^p  X  P(VP)  +  Uf-p  X  P(FP)  +  U^^  X  P(VN)  +  \}^^  x  P(FN) 

where  P(VP)   =  BR  x  SR  since  r    =  0 
P{FP)    =  SR  -  P(VP) 
P(FN)  =  BR  -  P(VP) 

P(VN)  =  (1  -  BR)  -  P(FP)  =  (1  -  SR)  -  P(FN)and 
U  J       _  0  since  no  testing  employed. 

Whenever  the  ROKAF  has  data  available  this  calculation  gives  the  ROKAF  a 
consistent  way  to  compare  alternatives  with  their  utilities.  For  the  present  effort,  it  is 
possible  to  simulate  ROKAF  data  and  evaluate  the  results.  Suppose  ROKAF  data  is 
as  follows: 
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SR  =  .6,  BR  =  .5,  N  =  500, 

utility  of  valid  positive,  in  dollar,  U^     =  S80,000, 
utility  of  false  positive,  L^    =  -  S80,000, 
utility  of  valid  negative,  U^^^  =  S50,000, 
utility  of  false  negative,  U^^^  =  0, 
Now  we  can  compute  the  expected  utility  of  a  random  selection  process  when 
the  ROKAF  uses  random  selection  of  KAFA  cadets; 

P{VP)   =  BR  X  SR    =  .5  X  .6  =  .3 

P(FP)    =  SR  -  P(VP)  =  .6  -  .3  =  .3 

P(FN)  =  BR  -  P(VP)  =  .5  -  .3  =  .2 

P(VN)  =  (1  -  SR)  -  P(FN)  =  (I  -  .6)  -  .2  =  .2 

then 

EU        =  80,000  X  .3  -  80,000  x  .3  +  50,000  x  .2  +  0  x  .2 
=  S10,000  per  candidate 

2.  Single-Stage  Decision  with  Medical  Examination 

When  a  medical  examination  is  employed  the  probability  of  valid  positive 
becomes: 

P(VP)  =  PS  X  SR 

where  PS  is  proportion  of  satisfactory  candidates  among  those  selected  by  a  test. 
Taylor  and  Russell  developed  a  table  for  this  PS  value  and  Appendix  C  provides  these 
values  as  different  BR  and  SR  which  is  similar  to  Taylor  and  Russell  tables  but  more 
appropriate  for  point-biserial  correlation,  r  ,  by  Abrahams,  Alf  and  Wolfe  [Ref  10].  If 
r   is  .25,  then  the  PS  =  .582  with  SR  =  .6,  BR  =  .5  at  Table  given  in  Appendix  C. 

Assuming  the  utility  of  medical  examination  is  S50  then  the  expected  utility 
becomes: 

P(VP)   =  PS  X  SR  =  .582  X  .6  =  .3492 
P(FP)    =  SR  -  P(VP)  =  .6  -  .3492  =  .2508 
P(FN)  =  BR  -  P(VP)  =  .5  -  .3492  =  .1746 
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P(VN)  =  (1  -  SR)  -  P(FN)  =  (1  -  .6)  -..1746  =  .2254 

thus, 

EU        =  80,000  X  .3492  -  80,000  x  .2508  +  50,000  x  .2254  -  50 
=  S  19,092  per  candidate 

which  is  more  than  the  previous  random  selection  with  the  amount  of  S9,092  utility 

using  only  a  medical  examination.    This  result  indicates  that  selection  via  the  medical 

examination  is  more  effective  than  random  selection.     In  other  words,  the  medical 

examination   increases   a   certain   amount   of  utility   of  possible   personnel   selection 

outcomes. 

3.  Double-Stage  Decision  with  Medical  and  an  Experimental  Test 

Since    the    terminal    decision    is    made    via    medical    and    experimental    test 

decisions,  it  is  reasonable  to  select  enough  candidates  at  the  first  decision  so  as  to  meet 

the  required  quota  at  the  second  decision.    In  the  second  decision,  the  terminal  decision 

is  made  concerning  the  provisionally  accepted  applicants  on  the  basis  of  the  first  test. 

If  we  assume  that  SR  =  .7  in  this  case  and  r^  =  .8  from  the  table  with  the  same  BR 

P 

then  the  value  of  PS  equals  .709.   With  the  utility  of  the  experimental  test  equal  to  S80, 
the  calculation  of  the  expected  utility  is,  therefore,  as  follows: 

P(VP)   =  PS  X  SR  =  .709  X  .7  =  .4963 

P(FP)    =  SR  -  P(VP)  =  .7  -  .4963  =  .2037 

P(FN)  =  BR  -  P(VP)  =  .5  -  .4963  =  .0037 

P(VN)  =  (I  -  SR)  -  P(FN)  =  {1  -  .7)  -  .0037  =  .2963 


then 


EU        =  80,000  X  .4963  -  80,000  x  .2037  +  50,000  x  .2963  -  50  -  80 
=  538,093  per  candidate 

which  is  more  than  three  times  the  random  selection  utility  and  more  than  SI 9,000 
utility  per  candidate  via  medical  test  only.  This  result  shows  that  if  the  ROKAF 
changes  the  present  pilot  selection  method  and  uses  an  experimental  test  method  with 
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medical  examination,  the  increased  expected  utility  of  that  change  is  about  S19,000  per 
candidate  even  if  using  small  scale  simulated  data. 

In  other  words,  the  potential  advantages  in  changing  the  present  ROKAF 
pilot  selection  system  are  obviously  considerable. 

F.       CAPER  MODEL  EVALUATION 

To  this  point  decision  strategies  have  been  discussed  in  terms  of  their  utilities  and 
advantages.  Managers  of  personnel  systems,  however,  justifiablely  demand  an  estimate 
of  the  payoff,  in  dollars,  which  can  be  expected  to  result  from  the  implementation  of  a 
proposed  selection  program.  It  might  appear  that,  for  a  given  test  validity  and  base 
rate,  a  personnel  manager  always  should  lower  the  selection  ratio  (i.e,  become  more 
selective)  this  strategy  may  or  may  not  be  cost  effective  since  lowering  selection  ratio 
causes  expanding  recruit  and  selection  efforts  to  meet  the  required  quota. 

The  purpose  of  this  section  is  again  to  evaluate  whether  the  changes  are  still 
feasible  in  terms  of  cost-effectiveness  by  using  the  Cost  of  Attaining  Personnel 
Requirements  (CAPER)  model.    Sands  [Ref  II]  defmed  the  CAPER  model  as  follows: 

•  "The  CAPER  model  is  designed  to  evaluate  the  cost  consequences  of  alternative 
recruiting-selection  strategies.  Specifically,  the  CAPER  model  determines  an 
optimal  recruiting-selection  strategy  for  minimizing  the  estimated  total  cost  of 
recruiting,  selecting,  inducting,  and  training  a  sufficient  number  of  persons  to 
meet  a  specified  quota  of  satisfactory  personnel.  In  addition,  the  CAPER 
model  considers  the  cost  of  an  erroneous  acceptance  (selecting  a  person  for  a 
training  program  who  subsequently  fails  to  graduate)  and  the  cost  of  erroneous 
rejection  (rejecting  a  person  who  would  have  succeeded  if  given  the 
opportunity)." 

Suppose  that  the  ROKAF  General  Staff  provides  specific  information  for  pilot 

selection   policy   such   that   the   quota,   the   base   rate   and   cost   data   for   selection, 

recruiting,  induction,  training,  erroneous  acceptance  and  rejections,  etc.    Example  of 

input  data  for  the  ROKAF  is  provided  in  Table  8  [Ref  13]  and  shows  that  500  of  1,000 

persons  graduated,  indicating  a  .500  base  rate.   Suppose  the  data  was  given  as  follows: 

•  the  cost  of  recruiting  an  individual  is  S4,000, 

•  the  cost  of  the  present  selection  procedure  (medical  examination)  is  SI50 
including  the  physicians'  salaries  and  laboratory  fees, 

•  the  cost  of  administering  and  scoring  the  experimental  test  is  SIOO, 

•  the  induction  cost  per  individual  is  estimated  as  SI 50, 

•  the  cost  for  training  is  S3, 500, 

•  the  cost  of  erroneous  acceptance  is  set  at  S  1,000,  and 

•  the  cost  of  erroneous  rejection  is  estimated  as  S800. 
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TABLES 
EXAMPLE  CAPER  MODEL  INPUT  DATA  FOR  ROKAF 


Cuting 
Score 

Graduates 

Failures 

Qualified  for  Acceptance 

Qualified  for  Acceptance 

Frequency 

Number 

Proportion 

Frequency 

Number 

Proportion 

0 

0 

500 

1.000 

0 

500 

1.000 

1 

0 

500 

1.000 

7 

500 

1.000 

2 

0 

500 

1.000 

10 

493 

.986 

3 

0 

500 

1.000 

17 

483 

.966 

4 

2 

500 

1.000 

29 

466 

.932 

5 

6 

498 

.996 

45 

437 

.874 

6 

11 

492 

.984 

58 

392 

.784 

7 

21 

481 

.962 

68 

334 

.668 

8 

36 

460 

.920 

72 

266 

.532 

9 

52 

424 

.848 

66 

194 

.388 

10 

66 

372 

.744 

52 

128 

.256 

11 

72 

306 

.612 

36 

76 

.152 

12 

68 

234 

.468 

21 

40 

.080 

13 

58 

166 

.332 

11 

19 

.038 

14 

45 

108 

.216 

6 

8 

.016 

15 

29 

63 

.126 

2 

2 

.004 

16 

17 

34 

.068 

0 

0 

.000 

17 

10 

17 

.034 

0 

0 

.000 

18 

7 

7 

.014 

0 

0 

.000 
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Given  the  simulated  data,  the  ROKAF  can  plug  into  the  CAPER  model  the 
equations  and  compute  costs  the  present  ROKAF  selection  process  and  the  new 
selection  process.  The  set  of  equations  is  used  once  for  each  possible  cutting  score  on 
the  new  selection  process.  A  cutting  score  of  eight  on  the  test  will  be  used  for 
illustration. 

Equation  (I)  gives  the  formula  for  estimating  the  number  of  applicants  who  must 
be  recruited  in  order  to  meet  the  quota  : 

NR  =  Q/  [  (BR)(PG.)  ]        BR  >  0,  PGj  >  0  (1) 

where   NR  is  the  number  recruited, 

Q  is  the  quota  of  satisfactory  personnel, 

BR  is  the  base  rate,  and 

PG.  is  the  proportion  of  graduates  who  would  qualify  for  acceptance  at  the  /th 
cutting  score  on  the  test. 

Substituting  the  data  of  the  present  ROKAF  selection  procedure  into  Equation 

(1)  gives: 

NRp  =  50/[(0.50)(1.00)]  =  100 
Similarly,  for  the  experimental  selection  procedure  (  /=  8): 

NRg  =  50/  [  (0.50)(0.92)  ]  =  109 

Equation  (2)  gives  the  formula  for  estimating  the  number  of  erroneous 
acceptances  : 

NEA  =  NR(1  -  BR)PFj  (2) 

where  NEA  is  the  number  of  erroneous  acceptances, 

PF.  is  the  proportion  of  failures  who  would  qualify  for  acceptance  at  the  /th 
cutting  score  on  the  test,  and  the  remaining  symbols  are  defined  above. 

Substituting  the  data  of  the  present  ROKAF  selection  procedure  into  Equation 

(2)  gives: 
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NEAp  =  (100)(1  -  0.50)(1.00)  =  50 
Similarly,  for  the  experimental  selection  procedure  (  /=  8): 

NEAg  =  {109)(1  -  0.50X0.532)  =  29 

Equation  (3)  gives  the  formula  for  estimating  the  number  of  erroneous  rejections: 

NER  =  (NR)(BR){1  -  Pa  )  (3) 

where  NER  is  the  number  of  erroneous  rejections,  and  the  remaining  symbols  are 
defined  above. 

Substituting  the  data  of  the  present  ROKAF  selection  procedure  into  Equation 

(3)  gives: 

NERp  =  (100){0.50)(1  -  1.00)  =  0 
Similarly,  for  the  experimental  selection  procedure  (  /=  8): 

NERg  =  (109)(0.50)(1  -  0.92)  =  4 

Equation  (4)  gives  the  formula  for  estimating  the  number  of  persons  who  will 
accepted  : 

NA  =  Q  +  NEA  (4) 

where  NA  is  the  number  accepted  and  the  others  are  the  same  as  above. 

Substituting  the  data  of  the  present  ROKAF  selection  procedure  into  Equation 

(4)  gives: 


NA    =  50  4-  50  =  100 
P 


Similarly,  for  the  experimental  selection  procedure  (  /=  8): 
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NAg  =  50  +  29  =■  "79 


The  total  cost  of  the  present  selection  procedure  and  the  experimental  selection 
procedure  should  be  calculated  by  separate  equation. 

Equation  (5a)  gives  the  formula  for  estimating  the  total  cost  of  employing  the 
present  selection  procedure  to  meet  the  quota  of  satisfactory  personnel  : 


TCp  =  [  (NRXCR)  ]  +  [  (NRXCP)  ]  +  [  (NA)(CI)  ] 

+  [  (NA)(CT)  ]  +  [  (NEA)(CEA)  +  (NER){CER)  ]  (5a) 


where  TC    is  the  total  cost  of  using  the  present  selection  procedure  to  meet  the  quota, 

CR  is  the  cost  of  recruiting  a  person, 

CP  is  the  cost  of  administering  the  present  procedure, 

CI  is  the  cost  of  inducting  a  person, 

CT  is  the  cost  of  training  a  person, 

CEA  is  the  cost  of  an  erroneous  acceptance, 

CER  is  the  cost  of  an  erroneous  rejection,  and  the  others  are  the  same  as  above. 
Substituting  the  data  of  the  present  ROKAF  selection  procedure  into  Equation 
(5a)  gives: 


TC    =  [  (100)(S4000)  ]  +  [  (100)($1500)  ]  +  [  (100)(S100) 
+  [  (100)(S35000)  ]  +  [  (50)(S8000)  +  (0)(S1500)  ] 
=  S3,619,120 


Equation  (5b)  gives  the  formula  for  estimating  the  total  cost  of  employing  the 
experimental  selection  procedure  to  meet  the  quota  : 

TC^  =  [  (NR)(CR)  ]  +  [  (NR)(CP)  ]  +  [  (NR)(CE)  ]  +  [  (NA)(CI)  ] 

+  [  (NA)(CT)  ]  +  [  (NEA)(CEA)  +  (NER)(CER)  ]  (5b) 

where  TC  is  the  total  cost  of  using  the  experimental  selection  procedure  to  meet  the 
quota,  CE  is  the  cost  of  administering  the  experimental  procedure,  and  the  others  are 
the  same  as  above. 

Using  Equation  (5b)  for  the  experimental  selection  procedure  (  /=  8)  gives: 
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TC^    =  [  (109)(S4000)  ]  +  [  (109)(S1500)  ]  +  [  (109)(S80)  ]  +  [  (79)(S100)  ] 
+  (  (79)(S35000)  ]  +  [  (29)(S8000)  +  (4)(S1500)  ] 
=  S75 1,920 

A  useful  program  for  various  data  values  is  provided  in  Appendix  B  and  creates  a 
sample  CAPER  model  output  data  for  ROKAF  in  Table  9  for  the  different  values. 
This  output  contains  five  types  of  information  : 

1)  number  recruited, 

2)  number  of  erroneous  acceptance, 

3)  number  of  erroneous  rejections, 

4)  number  accepted, 

5)  total  cost. 

These  five  estimates  may  suffice  for  many  personnel  program  managers.  And  also 
contains  cost  for  each  of  the  following: 

1)  recruiting 

2)  selection 

3)  induction 

4)  training,  and 

5)  erroneous  decisions. 

Table  9  shows  that  as  the  cutting  score  increases  the  subsequent  results  are  as 
follows  : 

1)  a  greater  number  of  persons  must  be  recruited, 

2)  a  smaller  number  of  persons  is  accepted, 

3)  the  number  of  erroneous  acceptances  decreases,  and 

4)  there  is  an  increase  in  erroneous  rejections. 

These  four  consequences  have  cost  complications.  As  recruiting  and  selection  costs 
increase,  induction  and  training  costs  are  decreased.  The  cost  of  erroneous  acceptances 
and  erroneous  rejections,  decreases  at  first,  hits  the  cutting  score  that  minimizes  the 
sum  of  both  costs  (  /=  11)  and  then  increases  as  the  cutting  score  is  raised  farther  and 
the  total  cost  changes  similarly.  Figure  4.4  graphically  shows  this  changes  of  total  cost 
as  increasing  the  cutoff  scores. 

If  the  ROKAF  wants  to  achieve  50  graduates  at  a  minimum  total  cost, 
comparing  the  estimated  total  cost  of  the  present  selection  procedure  (S4,460,000)  in 
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TABLE  9 
EXAMPLE  OF  CAPER  MODEL  OUTPUT  DATA  OF  ROKAF 


CUT 

NO 

NO 

NO 

NO 

SCORE 

REC. 

ACC. 

ERR. 
ACC. 

ERR. 
REJ. 

COST 

RECRUIT 

SELECT 

INDUCT 

TRAINING 

ERR. DEC. 

TOTAL 

NA 

100 

100 

50 

0 

400000. 

150000. 

10000. 

3500000. 

400000. 

4460000. 

0 

100 

100 

50 

0 

<*00000. 

158000. 

10000. 

3500000. 

400000. 

4468000. 

1 

100 

100 

50 

0 

-iOOOOO. 

158000. 

10000. 

3500000. 

400000. 

4468000 . 

2 

100 

99 

i.9 

0 

"400000. 

153000. 

9900. 

3465000. 

392000. 

4424900 . 

3 

100 

98 

<i8 

0 

"iOOOOO. 

158000. 

9800. 

3430000. 

384000. 

4381800. 

<i 

100 

97 

<i7 

0 

"iOOOOO. 

158000. 

9700. 

3395000. 

376000. 

4338700. 

5 

100 

9<i 

"i^ 

0 

"400000. 

158000. 

9400. 

3290000. 

352000. 

4209400. 

6 

102 

90 

^0 

1 

<iG8000. 

161160. 

9000. 

3150C00. 

321500. 

4049660 . 

7 

104 

35 

35 

2 

"416000. 

164320. 

8500. 

2975000. 

283000. 

3846820. 

8 

109 

79 

29 

<♦ 

436000. 

172220. 

7900. 

2765000. 

238000. 

3619120. 

9 

113 

73 

23 

9 

472000. 

■  136440. 

7300. 

2555000. 

197500. 

3413240. 

10 

13<i 

67 

17 

17 

536000. 

211720. 

6700. 

2345000. 

161500. 

3260920. 

U 

163 

62 

12 

32 

652000. 

257540. 

6200. 

2170000. 

144000. 

3229743. 

12 

21<i 

59 

9 

57 

856000. 

338120. 

5900. 

2065000. 

157500. 

342252G. 

13 

301 

56 

6 

101 

120"4000. 

475580. 

5600. 

1960000. 

199500. 

3844680. 

1"^ 

^6  3 

5<> 

4 

181 

1S52000. 

731540. 

5400. 

1890000. 

303500. 

4782440. 

15 

79^ 

52 

2 

i^7 

3176000. 

1254520. 

5200. 

1820000. 

536500. 

6792220. 

16 

lii?! 

50 

0 

685 

5884000. 

2324180. 

5000. 

1750000. 

1027500. 

10990680. 

17 

29<H 

50 

0 

l'i21 

11764000. 

4646780. 

5000. 

1750000. 

2131500. 

20297280. 

18 

71<i3 

50 

0 

3521 

28572000. 

11285940. 

5000. 

1750000. 

5281500. 

46894416. 

Table  9  suggest  that  using  the  experimental  testing  procedure  with  a  cutting  score  of  8 
(S3, 619, 120)  would  be  cost  effective  which  is  the  same  result  as  the  utility  evaluation. 

Table  9  shows  that  the  minimum  total  cost  of  attaining  the  quota,  50  graduates, 
is  S3, 229, 740  using  a  cutting  score  U  on  the  test  and  the  optimal  recruiting-selection 
strategy  is  to  recruit  163  persons.  The  best  estimate  is  that  62  of  these  persons  will 
qualify  for  acceptance  and  12  of  the  selectees  can  be  expected  to  fail  the  training 
course,  leaving  the  50  graduates  required  to  meet  the  required  quota.  In  comparison 
with  the  present  selection  procedure,  the  use  of  the  experimental  selection  procedure 
will  save  an  estimated  Si, 230,260  or  S24,605.20  per  graduate. 
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Figure  4.4    Graphical  view  of  CAPER  model  output. 
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V.  DISCUSSION 

The  increasing  number  of  pilot  candidates  entering  the  ROKAF  has  resulted  in 
increasing  costs  due  to  high  attrition  rates  and  decreasing  quality  of  pilots  who  do 
complete  the  program.  Further,  training  funds  and  personnel  resources  remain  limited. 
Solving  these  problems  will  not  be  an  easy  task.  It  appears  that  there  is  not  a  single 
simple  solution.  However,  it  is  imperative  that  some  restructuring  of  the  selection 
process  be  initiated  as  soon  as  possible  as  the  potential  negative  impact  these  problems 
will  have  continues  to  increase.  It  is  hoped  that  the  current  "Operations  with 
Economic  Efficiency  (OEE)  Program",  which  is  attempting  to  maximize  the  cost 
effectiveness  of  all  ROKAF  operations  will  be  able  to  adequately  address  the  pilot 
selection  problems.  This  thesis  represents  an  attempt  to  suggest  an  effective  method 
for  improving  the  selection  process. 

As  was  discussed  in  Chapter  I,  other  nations  -  France,  Italy  and  England  - 
initiated  the  selection  testing  efforts  as  early  as  World  War  I  and  the  USN  (USAF) 
reduced  the  attrition  rate  from  60%  to  30%  (25%)  using  psychological  testing  in  the 
early  1970s.  The  Pensacola  1000  aviator  study  found  that  psychological  and 
psychomotor  measures  had  more  validity  for  the  prediction  of  success  in  flight  training 
than  did  physiological  measures.  The  psychomotor  testing  scores  are  significantly 
related  to  flight  training  pass/fail  criteria.  The  ROKAF  pilot  selection  system, 
however,  only  depends  on  actual  flight  performance. 

As  we  evaluated  its  potential  benefits  to  the  ROKAF  in  Chapter  IV,  the  expected 
utilities  and  CAPER  model,  could  have  a  significant  impact  on  pilot  selection,  pilot 
training  and  aircraft  accidents.  In  terms  of  the  analytic  CAPER  model,  where  output 
data  are  fixed  by  the  values  of  the  input  data,  the  utility  of  the  CAPER  output  can  be 
no  better  than  the  accuracy  of  the  input  data.  If  the  cost  estimates  are  inaccurate,  cost 
forecasts  and  the  optimal  recruiting- selection  strategy  will  be  distorted.  This  may  not 
be  a  serious  problem  since  the  approximations  are  usually  adequate  for  the  kinds  of 
decisions  that  estimates  are  used  to  make,  and  there  is  no  need  for  utility  estimate 
down  to  the  last  dollar. 

It  has  been  shown  that  a  greater  number  of  pilot  candidates  attrite  during  actual 
flight    training    than    during    the    pre-training    medical    screening    process.     These 
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candidates,  though  often  having  the  option  to  continue  their  military  careers  in  other 
aviation  related  fields,  frequently  feel  that  they  are  inferior  officers  because  of  the 
failure  during  flight  training.  These  feelings  of  low  self  esteem,  morale  and  motivation 
often  result  in  poor  performance  and  the  involuntary  termination  of  the  officer's  career. 
It  is  suggested  that  if  a  viable  psychological  test  battery  was  administered  in  addition 
to  medical  screening  and  a  candidate  failed  that  test  then  (1)  there  would  be  a 
significant  savings  in  actual  flight  training  time  for  that  candidate  and  (2)  his  being 
identified  as  "not  aeronautically  adaptable"  might  not  have  the  psychological  impact  as 
having  failed  because  of  poor  performance  during  actual  training  flights. 

A.       RECOMMENDATIONS 

This  study  which  is  one  of  the  GEE  programs  planned  to  reduce  the  cost  and 
improve  the  efficiency  of  all  ROKAF  operations  was  essentially  designed  to  emphasize 
the  importance  and  necessity  of  mental  testing  methods  on  pilot  selection  for 
successfully  screening  out  potential  failures  and  suggests  useful  pilot  screening  methods 
to  ROKAF.  Also  provided  are  evaluation  techniques  for  decision  makers  which  are 
the  decision  strategies  and  CAPER  model. 

It  is  clear  from  the  overall  study  results  that  the  use  of  psychological  tests  or  test 
batteries  of  other  nations  can  improve  the  prediction  outcomes  of  the  ROKAF  pilot 
candidate  selection  method.  Many  other  national  Air  Forces  have  used  psychomotor 
screening  to  select  their  pilot  candidates  and  continue  efforts  to  develop  this 
methodology.  Some  of  the  psychomotor  scores  are  significantly  different  between 
candidates  who  graduate  from  UPT  and  those  who  are  eliminated.  The  ideal  selection 
system,  however,  would  measure  and  consider  all  relevant  characteristics  of  a 
candidate.  The  UPT  selection  system  currently  used  by  the  ROKAF  does  not  consider 
all  relevant  information  about  candidates  and  many  candidates  fail  training  because  of 
deficiencies  in  flying  aptitude.  The  intense  situation  of  ROKAF,  moreover,  requires 
selecting  high  quality  student  pilots  and  the  candidates  with  the  lowest  probability  of 
success  would  be  screened  out  before  training.  The  higher  the  cutoff  point  is  set,  the 
better  the  pilot  quality,  cost  effectiveness  not  with  standing.  The  "false  rejection", 
however,  is  a  problem  with  all  screening  systems,  including  flight  screening,  and  must 
be  evaluated  in  terms  of  the  percentage  screened  and  the  quality  of  the  graduating 
students. 

As  suggested  in  Chapter  IV,  a  change  using  experimental  testing  on  ROKAF 
pilot  selection  method  is  feasible.    It  suggests  a  savings  of  S24,605.20  per  graduate  in 
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our  example.  Multitask  performance  is  useful  .for  the  primary  flight  training 
performance  prediction.  The  experiment  takes  only  20  minutes  and  is  easy  to 
administer.  The  psychomotor  screening  system  used  by  the  USAF,  which  includes 
Two-Hand  Coordination  and  Complex  Coordination,  has  been  shown  to  be  useful  in 
screening  not  only  candidates  for  UPT  but  also  for  identifying  those  who  demonstrate 
a  potential  for  successfully  completing  advanced  fighter  training.  Any  of  three 
Integrated  Pilot  Candidate  Selection  Models  could  be  used  to  select  UPT  candidates  in 
the  same  manner  as  the  psychomotor  screening  system.  The  whole  person  concept  is 
imbedded  in  the  multidimensional  IPCSM  approach  to  pilot  selection. 

The  implementation  of  psychomotor  screening  and  multitask  performance  will 
improve  the  quality  of  student  candidates  and  reduce  attritions  in  ROKAF  UPT. 
Based  on  overall  considerations,  however,  an  integrated  selection  system  including 
psychomotor  screening  is  recommended  for  ROKAF. 

One  of  the  preferable  pre-selection  systems  that  suggests  for  ROKAF  pilot 
selection  system  of  AFA  cadets  in  this  study  would  be: 

1)  The  General  Staff  defines  the  number  of  pilots  needed,  and  a  circular  is  issued 
identifying  the  number  of  candidates  required. 

2)  The  selection  process  includes: 

a)  Preliminary  Examination  -  Medical 

b)  EEG  recordings  (Electroencephalogram) 

c)  Integrated  Psychological  Test 

d)  Individual  interview. 

3)  Cadets  must  wait  until  after  their  graduation  from  the  KAFA  and  commission 
as  officers  before  starting  flight  training. 

Other  useful  methods  which  might  also  be  incorporated  in  a  pre-selection  screening 

process  would  be: 

•  Perceptual/Cognitive  Paper-and- Pencil  Tests, 

•  Neurological  Test, 

•  Residual  Attention  Test. 

The  Residual  Attention  Test  could  be  especially  valuable  in  predicting  how  well  an 
individual  will  handle  emergency  situations.  This  could  potentially  have  an  impact  on 
the  aircraft  accident  rate  in  accidents  due  to  human  error,  incorrect  procedures  or  just 
poor  handling  of  the  aircraft  during  emergencies.  It  is  obvious  that  with  the  many 
types  of  individual  psychological  and  psychomotor  tests  available  that  further  research 
must  now  address  which  individual  tests  and  combinations  of  tests  and  batteries  of 
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must  now  address  which  individual  tests  and  conabinations  of  tests  and  batteries  of 
tests  should  be  incorporated  into  the  ROKAF  selection  program.  It  is  strongly 
recommended  that  a  ROKAF  pilot  selection  program  research  and  development  (R  & 
D)  team  be  organized  to  address  this  and  other  selection  issues.  The  team  could  also 
benefit  greatly  from  a  close  liaison  with  U.S.  Navy  and  Air  Force  R&D  centers 
currently  engaged  in  aircrew  selection  methodology. 

It  is  hoped  that  this  study  will  be  useful  in  stimulating  the  ROKAF  in  developing 
improved  selection  methods  and  in  predicting  success  in  training  as  a  means  of 
increasing  aircrew  training  program  cost  effectiveness. 
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List  of  Abbreviation 

ACTB  Aircrew  Classification  Test  Battery 

AFA  Air  Force  Academy 

AFB  Air  Force  Base 

AFHRL  Air  Force  Human  Resources  Laboratory  (AFSC) 

AFOQT  Air  Force  OfTicer  Qualifying  Test 

AFROTC  Air  Force  Reserve  Officer  Training  Corps 

AOC  Aviator  Officer  Candidate 

AQT  Aviation  Qualification  Test 

ASTB  Aviator  Selection  Test  Battery 

FAR  Flight  Aptitude  Rating 

FR  Fighter  Recommended 

FSP  Flight  Screening  Program  (OTS) 

IPCSM  Integrated  Pilot  Candidate  Selection  Model 

KAFA  Korea  Air  Force  Academy 

OEE  Operations  with  Economic  Efficiency 

OTS  Officer  Training  School 

ROKAF  Repubhc  of  Korea  Air  Force 

R&D  Research  and  Development 

UPT  Undergraduate  Pilot  Training 

USAF  United  States  Air  Force 

USN  United  States  Navy 
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APPENDIX  A 
ATTRITION  CATEGORIES 

A  student  pilot  can  be  dropped  from  a  training  program  for  several  reasons.  It  is 
agreed  that  there  is  no  one  simple  causative  factor  nor  a  quick  and  easy  solution. 
Some  of  the  primary  reasons  for  student  attrition  are  : 

/.    Voluntary  Withdrawal  or  Drop  On  Request  (DOR) 

Over  50  research  reports  and  findings  associate  the  main  reason  of  DOR 
with  anxiety,  tension  or  fear.  Most  of  the  specific  reasons  for  the  DOR  are  expressed 
an  anxiety  or  tension  with  regard  to  flying,  fear  of  flying,  or  lack  of  confidence  in 
handling  aircraft.  Also  mentioned  are  a  dislike  or  distaste  of  flying  and  a  shift  to  a 
civilian  career  interest.  Some  of  the  researches  indicate  a  connection  of  fear  of  aircraft 
accidents  or  crashing  with  failure  in  training. 
2.   Not  Phisically  Qualified 

Most  research  suggests  that  "a  person  with  anxiety  may  see  the  flight 
surgeon  when  he  has  a  symptom  which  he  knows  might  result  in  his  being  dropped 
from  the  program;  whereas,  a  nonanxious,  highly  motivated  person  might  never  go  to 
the  flight  surgeon." 

5.   Flight  Failure  (FF) 

Figure  1.1  mentions  the  types  of  attrition  by  stage  of  training.  The  data 
provide  dramatic  evidence  of  the  relationship  of  DOR  and  FF  in  primary  training  and 
the  transition  phase. 

The  research  literature  generally  supports  the  view  that  anxiety  is  both  a 
cause  and  a  symptom  of  inadequate  or  inferior  performance  in  training.  It  can  be 
broken  down  into  two  subcategories: 

1)  A  student  group  prone  to  anxiety  or  with  anxiety  predisposition. 

2)  A  student  group  going  through  a  transient  anxiety  producing  situation. 

4.  Not  Officer  Material 

Is  a  statistic  more  oriented  to  the  time  during  which  attrition  occurs  rather 
than  a  specific  type  of  attrition. 

5.  Academic  Failure 

Unsatisfactory  performance  in  academic  work. 
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6.  Transfer  to  Another  Training  Program    - 

This  is  a  small  percentage  of  the  total  attrition  and  means  a  request  to 
change  one's  specialty. 

7.  Practical  Work  Failure 

Unsatisfactory  performance  in  a  ground  simulator  or  training  device. 
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APPENDIX  B 
CAPER  MODEL  PROGRAM 


******************************************************************* 

*  * 

*  CAPER  MODEL  * 

*  * 

*  This  program  provides  a  CAPER  MODEL  OUTPUT  as  various  input  data.  * 

*  The  program  reads  data  from  data  file,  CAPER  DATA,  and  creates  a  * 

*  formated  output,  CAPEROU  LISTING  on  your  disk  automatically  using  * 

*  exec  file.   The  data  used  are  from  tne  Table  8  and  there  is  no  * 

*  no  meaning  of  ROKAF  data  but  only  for  showing  how  the  program  works.  * 

*  The  key  variables  used  are  •  * 


*  * 

number  recruited  * 

number  accepted  * 

number  of  erroneous  acceptances                  * 

number  of  erroneous  rejections                   * 

the  quota  of  satisfactory  personnel  * 

base  rate  * 


*  NR 

*  NA 

*  NEA 

*  NER 

*  NQ 


*  BR 

*  PG   :  proportion  of  graduates  who  would  qualify  for      * 

*  acceptance  at  the^ith  cutting  score  on^test        * 

* 
* 
* 

* 


*  CR 

*  CP 

*  CE 

*  CI 

*  CT 

*  CEA 

*  CER 

*  TCP 


* 


*  PF   :  proportion  of  failures  who  would  qualify  for 

*  __    acceptance  at  the  ith  cutting  score  on  test 

cost  of  recruiting  a  person 
cost  of  administering  the  present  procedure 
cost  of  administering  the  experimental  procedure 
cost  of  inducting  a  person  * 

cost  of  training  a  person  * 

cost  of  an  erroneous  acceptance  * 

cost  of  an  erroneous  rejection  * 

total  cost  of  using  the  present  selection         * 

*  procedure  to  meet  the  quota  * 

*  TCE  :  total  cost  of  using  the  experimental  selection      * 

*  procedure  to  meet  the  quota  * 

*  * 

*  This  program  could  be  used  effectively  for  other  CAPER  MODEL     * 

*  changing  only  the  data  and  also  useful  for  interactive  program  with  * 

*  some  developments.  * 

************************************************************************ 

PARAMETER  (N=20' 
REAL  PG(N),PF(N; 

DATA  NQ/50/,BR/.5/,CR/4000./,CP/1500./,CI/100./,CE/80./,CT/35000./ 
*     CEA/8000/,CER/1500/ 

***  READ  DATA  FROM  DATA  FILE  *** 

DO  10  1=1. N 

READ  (1,5)  PG(I),PF(I) 
10    CONTINUE 

***  OUTPUT  FORMAT  *** 

WRITE (3, 15 

WRITE(3,65, 
65    F0RMAT(1X,'   CUT  |   NO   |   NO   |   NO   |   NO  \',12X,'\') 

WRITE (3,75) 
75    FORMAT ( IX, '  I  SCORE  I  REC.  |  ACC.  |  ERR.  |  ERR.  |  ' , 34X, ' COST ' ,34X, ' | ' 

WRITE(3,*)  'I      I       I       I 


*)   r 


WRITE ( 3 

WRITE(3)*)' ' I      I       I       I  ACC.  I  REJ.  |     RECRUIT  |    SEL 
*ECT   I    INDUCT   I   TRAINING  |   ERR. DEC.  |    TOTAL    I' 
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WRITE (3, 15) 
WRITE(3,*) 

CALCULATE  TOTAL  COST  *** 

DO  20  J=1,N 

NR  =  NINT(NQ/(BR  *  PG(J))) 
NEA  =  NINT(NR  '^  (1  -  BR)  * 
NER  =  NINTCNR  *  BR  '  ' 


(1  -  PG( 


NA 

= 

UQ   + 

NEA 

TR 

= 

NR  * 

CR 

SP 

= 

NR  * 

CP 

SE 

= 

NR  * 

CP  + 

NR  * 

:e 

TI 

= 

NA  * 

CI 

TT 

= 

NA  * 

CT 

TE 

= 

NEA  *  CEA 

+  NER 

•k 

CER 

TCS 

■ 

=  TR  +  TI  +  TT  + 

TE 

^^iii! 


TCP  =  TCS  +  SP 
TCE  =  TCS  +  SE 
IF  (J.EQ.l)  THEN 

WRITE (3, 35)  NR,NA,NEA,NER,TR,SP,TI,TT,TE,TCP 

WRITE(3,55) 
ELSE 

WRITE(3,45)  (J-2) ,NR,NA,NEA,NER,TR,SE ,TI ,TT,TE ,TCE 
END  IF 


20 

CONTINUE 

WRITE(3,15) 
FORMAT  (F5.3,5X,F5.3) 

5 

15 

FORMAT  (IX, '+' ,106('  '),'+') 
FORMAT  ('+' ,T37,72('-' ),'!') 

25 

35 

FORMAT (IX, ' 

NA   1  ',4(14,'  1 

45 

FORI-IATflX,  ' 

M2,'   1  ',4(14 
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FORMAT (IX, ' 

',T8,'|  ',4(4X,' 

STOP 

END 

***  DATA  FILE  *** 

PG 

PF 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.986 

1.000 

.966 

1.000 

.932 

.996 

.874 

.984 

.784 

.962 

.668 

.920 

.532 

.848 

.388 

.744 

.256 

.612 

.152 

.468 

.080 

.332 

.038 

.216 

.016 

.126 

.004 

.068 

.000 

.034 

.000 

.014 

.000 

'),6(F10.0,'  I 
■  '  '),6(F10.0 


I  '),6(i0X,'  I'j) 


'  I')) 
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APPENDIX  C 
TAYLOR  AND  RUSSELL  TABLES 

N.  M.  Abrahams,  E.  F.  Alt,  and  J.  H.  Worn 

TABLE  1 
PxoTOKTiON  Wbo  Wol  Bx  Satstacioiy  Ajioiro  Tbosz  Sxixcizd,  701  GivxN  VAU7XS  or 

TSZ  PSOPOKTIOM  Ot  PUSXNT  EMTUYXXS  CoNSIDUZD  SaTBIACIOIY,  OB 
SxlXCtlOM  RaSO,  AMD  PoOfT-BlSZUAI.  f 


Seiecnon  rado 


.050 


100         JOO         JOO         .400         .iOO         .600         .700  .800         .900 


.950 


Proporooa  of  empioyeci  muidered  sadsfactory  «  .05 


.00 

.050 

.050 

.050 

.050 

.050 

.050 

.050 

.050 

.050 

.050 

.050 

.05 

.076 

.072 

.066 

.063 

.061 

.059 

.057 

.055 

.054 

.052 

.051 

.10 

.112 

.099 

.086 

.078 

.072 

.067 

.063 

.060 

.056 

.053 

.052 

.15 

.157 

.131 

.106 

.092 

.082 

.073 

.068 

.063 

.058 

.054 

.052 

.20 

.213 

.169 

.128 

.107 

.092 

.082 

.073 

.066 

.060 

.055 

.052 

J5 

27S 

.210 

.151 

.121 

.101 

.087 

.076 

.068 

.061 

.055 

.052 

.W 

J52 

.255 

.173 

.133 

.109 

.092 

.079 

.070 

.062 

.053 

.053 

.35 

.433 

.302 

.194 

.144 

.113 

.095 

.081 

.070 

.062 

.056 

.053 

.40 

J19 

.34* 

.212 

.153 

.119 

.097 

.082 

.071 

.062 

.056 

.053 

.45 

.607 

.393 

.227 

.159 

.122 

.099 

.083 

.071 

.062 

.056 

.053 

.50 

.694 

.431 

.238 

.163 

.124 

.100 

.083 

.071 

.062 

.056 

.053 

.55 

.776 

.462 

.245 

.165 

.124 

.100 

.083 

.071 

.062 

.056 

.053 

.60 

.849 

.484 

.248 

.166 

.125 

.100 

.083 

.071 

.062 

.056 

.053 

.65 

.909 

.495 

.250 

.167 

.125 

.100 

.083 

.071 

.062 

.056 

.053 

.70 

.954 

.499 

.250 

.167 

.125 

.100 

.083 

.071 

.062 

.056 

.053 

.73 

.982 

.500 

.250 

.167 

.125 

.100 

.083 

.071 

.062 

.056 

.053 

.80 

.996 

.500 

.250 

.167 

.125 

.100 

.083 

.071 

.062 

.056 

053 

.85 

.996 

.500 

.250 

.167 

.125 

.100 

.083 

.071 

.062 

.056 

.053 

.90 

.996 

.500 

J50 

.167 

.125 

.100 

.083 

.071 

.062 

.056 

.053 

.95 

1.000 

.500 

.250 

.167 

.125 

.100 

.083 

.071 

.062 

.056 

.053 

1.00 

1.000 

.500 

.250 

.167 

.125 

.too 

.083 

.071 

.062 

.056 

os; 

Proporooa  of  employees  considered  satisfactory  ■■  .10 


.00 
.05 
.10 
.15 
.20 
.25 
JO 
J5 
.40 
.45 
JO 
JS 
.60 
.65 
.70 
.75 
.80 
.85 
.90 
.95 
1.00 


.100 

.134 

.176 

.226 

.284 

J49 

.422 

J02 

.388 

.676 

.765 

J47 

.917 

.966 

.991 

.999 

1.000 

1.000 

1.000 

1.000 

1.000 


.100 
.129 
.162 
JOO 
.243 
.290 

J99 
.460 
.525 
J92 
.661 
.731 
.800 
.864 
.920 
.964 
.990 
.999 
.999 
1.000 


.100 

.122 

.147 

.173 

J02 

.232 

.264 

.297 

J30 

J64 

-397 

.427 

.454 

.476 

.490 

.497 

JOO 

JOO 

-SOO 

.500 

.500 


.100 

.118 

.137 

.157 

.177 

.198 

J19 

2*0 

J60 

.278 

J95 

J09 

J20 

J27 

J31 

J33 

J33 

.233 

-333 

.333 

J33 


.100 

J15 

.130 

.143 

.160 

.175 

.189 

J02 

.214 

.225 

.234 

J40 

.245 

J48 

J49 

J50 

J50 

J50 

.250 

.250 

J50 


.100 

.112 

.124 

.135 

.146 

.157 

.166 

.175 

.182 

.188 

.193 

.196 

.198 

.199 

.200 

JOO 

JOO 

JOO 

JOO 

JOO 

JOO 


.100 

.110 

.118 

.127 

.135 

.142 

.148 

.153 

.158 

.161 

.164 

.165 

.166 

.166 

.167 

.167 

.167 

.167 

.167 

.167 

.167 


.100 

.107 

.114 

.120 

.123 

.130 

.133 

.136 

.139 

.140 

.142 

.142 

.143 

.143 

.143 

.143 

.143 

.143 

.143 

.143 

.143 


.100 

.105 

.109 

.113 

.116 

.119 

.121 

.122 

.123 

.124 

.125 

.125 

.125 

.125 

.125 

.125 

.125 

.125 

.125 

.125 

.125 


.100 

.103 

.105 

.107 

.108 

.109 

.110 

.110 

.111 

.111 

.111 

.111 

.111 

.111 

.111 

.111 

.111 

.111 

.111 

.111 

.111 


.100 
.102 
.103 
.104 
.104 
.104 
.105 

.la^ 

.lOJ 
.105 
.105 
.105 
.105 
.105 
.105 
.105 
.105 
.105 
.105 
.105 
.105 
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Taylor- RussEii  Tables  for  Dichotomous  -Criterion  Variables 


TABLE  l—{C<mtinuai) 


rp» 


Selection  ratio 


.050 


.100  JOO 


JOO 


.400 


500  .600 


.700 


.800 


.900 


.950 


Proportion  of  employees  considered  sati3factor>-  -  .20 


.00 

.200 

.200 

.200 

.200 

.200 

.200 

.200 

.200 

.200 

.200 

.200 

.05 

.244 

.237 

.229 

.224 

.220 

.216 

.213 

.210 

.207 

.204 

.202 

.10 

.293 

.277 

.260 

.248 

.239 

.232 

.225 

.219 

.213 

.207 

.204 

.15 

.347 

.320 

.292 

.274 

.260 

.248 

.238 

.228 

.219 

.210 

J205 

.20 

.405 

.368 

.326 

.300 

.280 

.264 

.249 

.236 

.224 

.213 

.207 

.25 

.468 

.418 

.363 

.328 

.301 

.280 

.261 

.244 

.229 

.215 

.208 

.30 

.535 

.472 

.401 

.356 

.322 

.295 

.272 

.252 

.234 

.217 

.208 

.35 

.60S 

.529 

.441 

.385 

.344 

.311 

.283 

.259 

.238 

.218 

.209 

.40 

.676 

.589 

.484 

.416 

.366 

.326 

.293 

.265 

.241 

.220 

.210 

.45 

.747 

.652 

.529 

.448 

.387 

.340 

.302 

.270 

.244 

.220 

.210 

.50 

.815 

.717 

.577 

.480 

.409 

.354 

.310 

.275 

.246 

.221 

.    .210 

.55 

.876 

.782 

.627 

.514 

.430 

.367 

.318 

.279 

.248 

.222 

.210 

.60 

.927 

.846 

.680 

.547 

.450 

.378 

.324 

.282 

.249 

.222 

.210 

.65 

.964 

.904 

.736 

.581 

.468 

.387 

.328 

.284 

.249 

.222 

.210 

.70 

.986 

.952 

.794 

.612 

.482 

.394 

.331 

.285 

.250 

.222 

.210 

.75 

.996    1 

.983    ' 

.852 

.639 

.492 

.398 

.333 

.286 

.250 

.222 

.210 

.80 

1.000 

.997  ; 

.909 

.657 

.498 

.400 

.333 

.286 

.250 

.222 

.210 

35 

1.000 

1.000 

.958 

.665 

.500 

.400 

.333 

.286 

.250 

.222 

.210 

.90 

1.000 

1.000 

.990 

.667 

.500 

.400 

.333 

.286 

.250 

.222 

.210 

.95 

1.000    ' 

1.000 

.990 

.667 

.500 

.400 

.333 

.286 

.250 

.222 

.210 

1.00  ' 

1 

1.000    1 

1 

1.000    1 

1.000 

.667 

.500 

.400 

.333 

.286 

.250 

.222 

.211 

Proportion  ol 

employees  considered  satisfactory  =  .30 

.00 

.300 

.300 

'      .300 

.300 

.300 

.300 

.300 

.300 

.300 

.300 

300 

.05 

.349 

.341 

.333 

.327 

'      .322 

.318 

.315 

.311 

.308 

.304 

.302 

.10 

.401 

.385 

.367          .354 

.345 

.337 

.329 

.322 

.316 

.308 

.305 

.15 

.456 

.430 

.402    !      .383 

.368 

.355 

.344 

.333 

.323 

.312 

.307 

.20 

.513 

.478 

.438 

.412 

.391 

.374 

.358 

.344 

.330 

.316 

.308 

.25 

.572 

.528 

.476 

.442 

.415 

.393 

.372 

.354 

.336 

.319 

.310 

30 

.632 

.579 

.516 

.473 

.440 

.412 

.387 

.364 

.343 

.322 

.311 

35 

.692 

.632 

.557 

.506 

.465 

.431 

.401 

.374 

.349 

.324 

.312 

.40 

.751 

.686 

.600 

.540 

.492 

.451 

.415 

.383 

.354 

.327 

.314 

.45 

.808 

.740 

.646 

.576 

.519 

.471 

.429 

.392 

.359 

.329 

.314 

.50 

.860 

.794 

.693 

.614 

.547 

.491 

.443 

.401 

.363 

.330 

J15 

.55 

.905 

.346 

.742 

.653 

.577 

.512 

.456 

.408 

.367 

.331 

.315 

.60 

.942 

.894 

.794 

.696 

.608 

.532 

.468 

.415 

.370 

.332 

316 

.65 

.970 

.936 

.846 

.741 

.639 

.552 

.479 

.420 

.372 

.333 

.316 

.70 

.987 

.967 

.897 

.789 

.671 

.570 

.488 

.424 

.374 

.333 

.316 

.75 

.996 

.988 

.944 

.839 

.702 

.584 

.495 

.427 

.375 

.333 

.316 

.80 

.999 

.997 

.979 

.891 

.728 

.594 

.498 

.428 

.375 

.333 

.316 

.85 

1.000 

1.000 

.997 

.941 

.745 

.599 

.500 

.428 

.375 

.333 

.316 

.90 

1.000 

1.000 

l.OOO 

.982 

.750 

.600 

.500 

.429 

.375 

.333 

.316 

.95 

1.000 

1.000 

1.000 

.982 

.750 

.600 

.500 

.429 

.375 

.333 

.316 

1.00 

1.000 

1.000 

1.000 

1.000 

.750 

.600 

.500 

.429 

.375 

.333 

.316 

Proportion  of 

employees  considered  satisfactory  «  .40 

.00 

.400 

.400 

.400 

.400 

.400 

.400 

.400 

.400 

.400 

.400 

.400 

.05 

.451 

.444 

.435 

.429 

.424 

.420 

.416 

.412 

.408 

.405 

.403 

.10 

.504 

.488 

.470 

.458 

.448 

.439 

.432 

.424 

.417 

.409 

.405 

.15 

.557 

.534 

.506 

.487 

.472 

.459 

.447 

.436 

.425 

.414 

.408 

.20 

.611 

.580 

.542 

.517 

.497 

.479 

.463 

.US 

.433 

*" 

.410 
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^Pb 


Selection  ratio 


.050 


.100 


JOO 


.300 


.400 


.500 


.600 


.700 


.800 


.900 


.950 


Proporti 

on  of  em 

ployeea  considered 

satisfactory  ■  .40 

.25 

.665 

.626 

.580 

.548 

.522 

.500 

.479 

.460 

.441 

.422 

.412 

.30 

.717 

.673 

.618 

.580 

.548 

.521 

.496 

.472 

.449 

.426 

.414 

.35 

.768 

.720 

.658 

.613 

.575 

.542 

.512 

.484 

.456 

.429 

.415 

.40 

.816 

.766 

.698 

.647 

.603 

.564 

.529 

.496 

.464 

.433 

.417 

.45 

.860 

.811 

.740 

.682 

.632 

.588 

.546 

.508 

.471 

.436 

.418 

.50 

.900 

.854 

.782 

.720 

.663 

.612 

.564 

.519 

.477 

.438 

.419 

.55 

.933 

.894 

.824 

.758 

.696 

.637 

.582 

.530 

.483 

.440 

.420 

.60 

.959 

.929 

.866 

.799 

.730 

.663 

.600 

.541 

.489 

.442 

.420 

.    .65 

.978 

.958 

.906 

.842 

.768 

.691 

.617 

.551 

.493 

.443 

.421 

.70 

.990 

.979 

.943 

.885 

.807 

.719 

.634 

.560 

.496 

.444 

.421 

.75 

.997 

.992 

.972 

.928 

.850 

.748 

.649 

.566 

.498 

.444 

.421 

.80 

.999 

.998 

.991 

.966 

.894 

.774 

.660 

.570 

.500 

.444 

.4:. 

.85 

1.000 

1.000 

.998 

.992 

.939 

.792 

.665 

.571 

.500 

.444 

.421 

.90 

1.000 

1.000 

1.000 

1.000 

.979 

.800 

.667 

.571 

.500 

.444 

.421 

.95 

1.000 

1.000 

1.000 

1.000 

.979 

.800 

.667 

.571 

.500 

.444 

.421 

1.00 

1.000 

1.000 

1.000 

1.000 

1.000 

.800 

.667 

.571 

.500 

.444 

.421 

Proportion  of 

employees  considered  satisfactory  »  .50 

.00 

.500 

.500 

.500 

.500 

.500 

.500 

.500 

.500 

.500 

.500 

.500. 

.05 

.552 

.544 

.535 

.529 

.524 

.520 

.516 

.512 

.509 

.505 

.503 

.10 

.602 

.588 

.570 

.558 

.548 

.540 

.532 

.525 

.518 

.510 

.505 

.15 

.653 

.631 

.605 

.587 

.573 

.560 

.549 

.538 

.526 

.515 

.51.'% 

.20 

.702 

.674 

.640 

.617 

.598 

.581 

.565 

.550 

.535 

.519 

.5.1 

.25 

.748 

.716 

.676 

.647 

.623 

.602 

.582 

.563 

.544 

.524 

.513 

.30 

.793 

.757 

.712 

.678 

.649 

.623 

.599 

.576 

.553 

.529 

.515 

.35 

.834 

.797 

.747 

.709 

.676 

.646 

.617 

.590 

.562 

.533 

.518 

.40 

.872 

.835 

.783 

.741 

.703 

.669 

.636 

.603 

.571 

.537 

.520 

.45 

.905 

.871 

.818 

.773 

.732 

.693 

.655 

.617 

.580 

.541 

.521 

.50 

.933 

.903. 

.853 

.807 

.762 

.718 

.675 

.631 

.588 

.545 

.523 

.55 

.956 

.932 

.886 

.840 

.793 

.745 

.696 

.646 

.597 

.548 

.524 

.60 

.974 

.956 

.918 

.875 

.826 

.773 

.718 

.660 

.604 

.551 

.525 

.65 

.986 

.975 

.946 

.908 

.861 

.804 

.740 

.675 

.612 

.553 

.526 

.70 

.994 

.988 

.969 

.940 

.896 

.836 

.764 

.689 

.617 

.554 

.526 

.75 

.998 

.995 

.986 

.968 

.932 

.872 

.788 

.700 

.622 

.555 

.526 

.80 

1.000 

.999 

.996 

.988 

.966 

.909 

.810 

.709 

.624 

.555 

..=26 

.85 

1.000 

1.000 

.999 

.998 

.990 

.947 

.827 

.713 

.625 

.556 

.5.6 

.90 

1.000 

1.000 

1.000 

1.000 

.999 

.980 

.833 

.714 

.625 

.556 

.5^6 

.95 

1.000 

1.000 

1.000 

1.000 

1.000 

.999 

.833 

.714 

.625 

.556 

.526 

1.00 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.833 

.714 

.625 

.556 

.526 

Proportion  of 

employees  considered  satisfactory  «  .60 

.00 

.600 

.600 

.600 

.600 

.600 

.600 

.600 

.600 

.600 

.600 

.600 

.05 

.650 

.642 

.634 

.628 

.624 

.620 

.616 

.612 

.609 

.605 

.603 

.10 

.697 

.684 

.667 

.656 

.647 

.639 

.632 

.625 

.618 

.610 

.606 

.IS 

.742 

.723 

.700 

.684 

.671 

.659 

.648 

.637 

.626 

.615 

.608 

JO 

.784 

.761 

.733 

.712 

.695 

.679 

.664 

.650 

.636 

.620 

.611 

.25 

.824 

.798 

.764 

.740 

.719 

.700 

.681 

.663 

.645 

.625 

.614 

JO 

.859 

.832 

.796 

.768 

.743 

.721 

.699 

.677 

.655 

.630 

.617 

.35 

.891 

.864 

.826 

.796 

.768 

.742 

.717 

.691 

.664 

.636 

.619 

.40 

.918 

.893 

.855 

.823 

.793 

.764 

.735 

.706 

.675 

.641 

.622 

.45 

.942 

.920 

.883 

.851 

.819 

.788 

.755 

.721 

.685 

.646 

.624 
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TABLE  I— (Continued) 


Tpb 


Selection  ratio 


.050 


.100 


.200 


JOO 


.400 


.500 


.600 


.700 


.800 


.900 


.950 


Proportion  of  em 

ployees  considered 

satisfactory  »  .60 

.50 

.961 

.942 

.910 

.878 

.846 

.812 

.776 

.737 

.695 

.650 

.626 

.55 

.976 

.961 

.934 

.905 

.872 

.837 

.797 

.754 

.706 

.655 

.628 

.60 

.986 

.976 

.955 

.930 

.900 

.863 

.820 

.771 

.716 

.659 

.629 

.65 

.993 

.987 

.972 

.953 

.926 

.891 

.845 

.789 

.727 

.662 

.630 

.70 

.997 

.994 

.986 

.972 

.951 

.919 

.872 

.808 

.736 

.664 

.631 

.75 

.999 

.998 

.994 

.987 

.973 

.948 

.900 

.826 

.743 

.666 

.631 

,30 

1.000 

1.000 

.998 

.996 

.990 

.974 

.930 

.843 

.748 

.666 

.632 

.85 

1.000 

1.000 

1.000 

.999 

.998 

.992 

.960 

.854 

.750 

.667 

.632 

.90 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.986 

.857 

.750 

.667 

.632 

.95 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.999 

.857 

.750 

.667 

.632 

1.00 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.857 

.750 

.667 

.632 

Proportion  of 

employees  considered  satisfactory  =«  .  70 

.00 

.700 

.700 

.700 

.700 

.700 

.700 

.700 

.700 

.700 

.700 

.700 

.05 

.746 

.739 

.732 

.726 

.722 

.718 

.715 

.712 

.708 

.705 

.703 

.10 

.787 

.776 

.762 

.752 

.744 

.737 

.730 

.723 

.717 

.709 

.705 

.15 

.826 

.810 

.791 

.777 

.766 

.755 

.745 

.736 

.725 

.714 

.708 

.20 

.860 

.842 

.819 

.802 

.787 

.774 

.761 

.748 

.735 

.720 

.711 

25 

.890 

.871 

.846 

.826 

.809 

.793 

.777 

.761 

.744 

.725 

.714 

.30 

.916 

.897 

.871 

.849 

.830 

.812 

.793 

.774 

.754 

.731 

.718 

.35 

.938 

.920 

.894 

.872 

.852 

.831 

.810 

.788 

.764 

.737 

.721 

.40 

.956 

.941 

.916 

.894 

.873 

.851 

.828 

.803 

.775 

.743 

.724 

.45 

.970 

.958 

.936 

.915 

.894 

.871 

.846 

.818 

.786 

.749 

.727 

.50 

.981 

.972 

.953 

.935 

.914 

.891 

.865 

.834 

.798 

.755 

.730 

.55 

.989 

.982 

.968 

.952 

.934 

.912 

.885 

.851 

.811 

.761 

.732 

.60 

.994 

.990 

.980 

.968 

.952 

.932 

.905 

.870 

.823 

.766 

.734 

.65 

.997 

.995 

.989 

.981 

.969 

.952 

.926 

.889 

.836 

.771 

.735 

.70 

.999 

.998 

.995 

.990 

.982 

.970 

.948 

.909 

.849 

.774 

.736 

.75 

1.000 

1.000 

1.000 

.996 

.992 

.984 

.968 

.931 

.861 

.776 

.737 

,80 

1.000 

1.000 

l.OOO 

.999 

.998 

.994 

.986 

.953 

.870 

.778 

.737 

,85 

1.000 

1.000 

1.000 

1.000 

1.000 

.999 

.997 

.975 

.874 

.778 

.737 

.90 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.992 

.875 

.778 

.737 

.95 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.992 

.875 

.778 

.737 

1.00 

1.000 

1.000 

l.OOO 

1.000 

1.000 

1.000 

1.000 

1.000 

.875 

.778 

.737 

Proportion  of 

employees  considered  satisfactory  =  .i 

50 

.00 

.800 

.800 

.800 

.800 

.800 

.800 

.800 

.800 

.800 

.800 

.800 

.05 

.839 

.833 

.827 

.823 

.819 

.816 

.813 

.810 

.807 

.804 

.802 

.10 

.872 

.864 

.852 

.844 

.838 

.832 

.826 

.821 

.815 

.809 

.805 

.15 

.901 

.890 

.876 

.865 

.856 

.848 

.840 

.832 

.823 

.813 

.808 

.20 

.926 

.914 

.898 

.885 

.874 

.864 

.853 

.843 

.832 

.819 

.811 

.25 

.946 

.934 

.917 

.904 

.891 

.880 

.867 

.855 

.841 

.824 

.814 

.30 

.962 

.951 

.935 

.921 

.908 

.895 

.882 

.867 

.850 

.830 

.818 

.35 

.974 

.965 

.950 

.937 

.924 

.911 

.896 

.879 

.860 

.836 

.821 

.40 

.983 

.976 

.964 

.952 

.939 

.926 

.910 

.892 

.871 

.843 

.825 

.45 

.990 

.984 

.975 

.964 

.953 

.940 

.925 

.906 

.882 

.850 

.829 

.50 

.994 

.991 

.984 

.975 

.966 

.954 

.939 

.920 

.894 

.857 

.832 

.55 

.997 

.995 

.990 

.984 

.977 

.967 

.953 

.934 

.907 

.865 

.836 

.60 

.999 

.998 

.995 

.991 

.985 

.978 

.966 

.949 

.920 

.872 

.838 

.65 

1.000 

1.000 

1.000 

1.000 

.992 

.987 

.978 

.963 

.934 

.878 

.840 

.70 

1.000 

1.000 

1.000 

1.000 

1.000 

.994 

.988 

.977 

.948 

.884 

.841 
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rp6 


Selection  ratio 


.050 


.100  JOO 


JOO 


.400  JOO 


.600  .700 


.800 


.900 


.950 


Proportion  of  employees  conaidered  satisfactory  »  .80 


.75 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.995 

.988 

.963 

.887 

.842 

.80 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.999 

.996 

.977 

.888 

.842 

.85 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.999 

.990 

.889 

.842 

.90 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.998 

.889 

.842 

.95 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.998 

.889 

.842 

1.00 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.889 

.842 

Proportion  of 

employees  considered  satisfactory  -  .90 

.00 

.900 

.900 

.900 

.900 

.900 

.900 

.900 

.900 

.900 

.900 

.900 

.05 

.928 

.924 

.920 

.917 

.914 

.912 

.910 

.908 

.906 

.903 

.902 

.10 

.949 

.944 

.937 

.932 

.928 

.924 

.920 

.916 

.912 

.907 

.904 

.15 

.966 

.960 

.952 

.946 

.940 

.935 

.930 

.924 

.918 

.911 

.907 

.20 

.978 

.972 

.965 

.958 

.952 

.946 

.940 

.933 

.925 

.916 

.910 

.25 

.986 

.982 

.975 

.969 

.963 

.957 

.950 

.942 

.933 

.921 

.913 

.30 

.992 

.989 

.983 

.978 

.972 

.966 

.959 

.951 

.941 

.927 

.917 

.35 

.996 

.993 

.989 

.985 

.980 

.975 

.968 

.960 

.949 

.933 

.921 

.40 

.998 

.996 

.994 

.990 

.987 

.982 

.976 

.968 

.958 

.940 

.926 

.45 

.999 

.998 

.996 

.994 

.992 

.988 

.983 

.976 

.966 

.947 

.930 

.50 

1.000 

1.000 

1.000 

1.000 

.995 

.993 

.989 

.984 

.974 

.955 

.935 

.55 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.994 

.990 

.982 

.962 

.939 

.60 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.994 

.989 

.970 

.943 

.65 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.994 

.978 

.946 

.70 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.998 

.985 

W 

.75 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.999 

.991 

.947 

.80 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 
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